2023-10-06T04:17:12.368 INFO:root:teuthology version: 0.0.1.dev194+g54e62bc 2023-10-06T04:17:12.368 DEBUG:teuthology.run:Teuthology command: teuthology --owner scheduled_yuriw@teuthology --name yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi --archive /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270 --verbose --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} -- /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/orig.config.yaml 2023-10-06T04:17:12.412 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T04:17:12.496 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270 branch: wip-yuri-testing-2023-10-05-0730-quincy description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} email: yweinste@redhat.com first_in_suite: false job_id: '7412270' kernel: kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 8.stream overrides: admin_socket: branch: wip-yuri-testing-2023-10-05-0730-quincy ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 mds: debug mds: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - bad backtrace - object missing on disk - error reading table object - error reading sessionmap - unmatched fragstat - unmatched rstat - was unreadable, recreating it now - Scrub error on inode - Metadata damage detected - MDS_FAILED - MDS_DAMAGE sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 selinux: whitelist: - scontext=system_u:system_r:logrotate_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-yuri-testing-2023-10-05-0730-quincy sha1: f1e315a9732d36c05ec316678e2397dc58561c7f owner: scheduled_yuriw@teuthology priority: 75 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 7519 sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 sleep_before_teardown: 0 subset: 111/12000 suite: fs suite_branch: wip-yuri-testing-2023-10-05-0730-quincy suite_path: /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph-ci.git suite_sha1: f1e315a9732d36c05ec316678e2397dc58561c7f targets: smithi123.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNp26BaAn7SY4GNBkQ7fC3roi2Fy5jDHELVp1aQAnmwKkLpQ6giX1JCjpJCFE1YDQ/lLsVn0wDCnUbMkRM0TRoc= smithi177.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB2u8XPPwgdHORuEefQ8WR+QZ56B5fuFI/t8L6agrYh8Ue1xj0j0+zirQ3XAVbygcIz91+KnMCvpSIxGAfzXA54= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - ceph: null - ceph-fuse: null - cephfs_test_runner: modules: - tasks.cephfs.test_recovery_pool teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return false end - local function is_kdistro() return false end - local function is_fuse() return true end - local function syntax_version() return '' end teuthology_branch: main teuthology_sha1: 54e62bcbac4e53d9685e08328b790d3b20d71cae timestamp: 2023-10-05_23:52:50 tube: smithi user: yuriw verbose: true worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2830 2023-10-06T04:17:12.496 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa; will attempt to use it 2023-10-06T04:17:12.497 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa/tasks 2023-10-06T04:17:12.497 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-10-06T04:17:12.499 INFO:teuthology.task.internal:Checking packages... 2023-10-06T04:17:12.521 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2' 2023-10-06T04:17:12.522 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-10-06T04:17:12.522 INFO:teuthology.packaging:ref: None 2023-10-06T04:17:12.522 INFO:teuthology.packaging:tag: None 2023-10-06T04:17:12.522 INFO:teuthology.packaging:branch: wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T04:17:12.522 INFO:teuthology.packaging:sha1: 736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T04:17:12.522 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&ref=wip-yuri-testing-2023-10-05-0730-quincy 2023-10-06T04:17:12.666 INFO:teuthology.task.internal:Found packages for ceph version 17.2.6-1296.gf1e315a9 2023-10-06T04:17:12.666 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-10-06T04:17:12.673 INFO:teuthology.task.internal:no buildpackages task found 2023-10-06T04:17:12.673 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-10-06T04:17:12.748 INFO:teuthology.task.internal:Saving configuration 2023-10-06T04:17:12.764 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-10-06T04:17:12.793 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:17:12.820 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:12:16.140109', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNp26BaAn7SY4GNBkQ7fC3roi2Fy5jDHELVp1aQAnmwKkLpQ6giX1JCjpJCFE1YDQ/lLsVn0wDCnUbMkRM0TRoc='} 2023-10-06T04:17:12.844 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:12:16.141973', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB2u8XPPwgdHORuEefQ8WR+QZ56B5fuFI/t8L6agrYh8Ue1xj0j0+zirQ3XAVbygcIz91+KnMCvpSIxGAfzXA54='} 2023-10-06T04:17:12.845 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-10-06T04:17:12.851 INFO:teuthology.task.internal:roles: ubuntu@smithi123.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-10-06T04:17:12.851 INFO:teuthology.task.internal:roles: ubuntu@smithi177.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-10-06T04:17:12.851 INFO:teuthology.run_tasks:Running task console_log... 2023-10-06T04:17:12.956 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fc95393fb80>, signals=[15]) 2023-10-06T04:17:12.956 INFO:teuthology.run_tasks:Running task internal.connect... 2023-10-06T04:17:12.963 INFO:teuthology.task.internal:Opening connections... 2023-10-06T04:17:12.963 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:17:12.965 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:17:13.030 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:17:13.031 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:17:13.100 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-10-06T04:17:13.107 DEBUG:teuthology.orchestra.run.smithi123:> uname -m 2023-10-06T04:17:13.130 INFO:teuthology.orchestra.run.smithi123.stdout:x86_64 2023-10-06T04:17:13.131 DEBUG:teuthology.orchestra.run.smithi123:> cat /etc/os-release 2023-10-06T04:17:13.193 INFO:teuthology.orchestra.run.smithi123.stdout:NAME="CentOS Stream" 2023-10-06T04:17:13.193 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION="8" 2023-10-06T04:17:13.193 INFO:teuthology.orchestra.run.smithi123.stdout:ID="centos" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:ID_LIKE="rhel fedora" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:VERSION_ID="8" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:ANSI_COLOR="0;31" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:HOME_URL="https://centos.org/" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:17:13.194 INFO:teuthology.orchestra.run.smithi123.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T04:17:13.195 INFO:teuthology.lock.ops:Updating smithi123.front.sepia.ceph.com on lock server 2023-10-06T04:17:13.219 DEBUG:teuthology.orchestra.run.smithi177:> uname -m 2023-10-06T04:17:13.242 INFO:teuthology.orchestra.run.smithi177.stdout:x86_64 2023-10-06T04:17:13.242 DEBUG:teuthology.orchestra.run.smithi177:> cat /etc/os-release 2023-10-06T04:17:13.303 INFO:teuthology.orchestra.run.smithi177.stdout:NAME="CentOS Stream" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION="8" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:ID="centos" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:ID_LIKE="rhel fedora" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:VERSION_ID="8" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:PLATFORM_ID="platform:el8" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:PRETTY_NAME="CentOS Stream 8" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:ANSI_COLOR="0;31" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:CPE_NAME="cpe:/o:centos:centos:8" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:HOME_URL="https://centos.org/" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 8" 2023-10-06T04:17:13.304 INFO:teuthology.orchestra.run.smithi177.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2023-10-06T04:17:13.305 INFO:teuthology.lock.ops:Updating smithi177.front.sepia.ceph.com on lock server 2023-10-06T04:17:13.327 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-10-06T04:17:13.336 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-10-06T04:17:13.366 INFO:teuthology.task.internal:Checking for old test directory... 2023-10-06T04:17:13.366 DEBUG:teuthology.orchestra.run.smithi123:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:17:13.369 DEBUG:teuthology.orchestra.run.smithi177:> test '!' -e /home/ubuntu/cephtest 2023-10-06T04:17:13.388 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-10-06T04:17:13.395 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-10-06T04:17:13.395 DEBUG:teuthology.orchestra.run.smithi123:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:17:13.430 DEBUG:teuthology.orchestra.run.smithi177:> test -z $(ls -A /var/lib/ceph) 2023-10-06T04:17:13.481 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-10-06T04:17:13.538 INFO:teuthology.run_tasks:Running task kernel... 2023-10-06T04:17:13.555 INFO:teuthology.task.kernel:normalize config orig: {'kdb': True, 'sha1': 'distro'} 2023-10-06T04:17:13.555 INFO:teuthology.task.kernel:config {'mon.a': {'kdb': True, 'sha1': 'distro'}, 'mon.b': {'kdb': True, 'sha1': 'distro'}}, timeout 300 2023-10-06T04:17:13.556 DEBUG:teuthology.orchestra.run.smithi123:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:17:13.556 DEBUG:teuthology.orchestra.run.smithi177:> test -f /run/.containerenv -o -f /.dockerenv 2023-10-06T04:17:13.575 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:17:13.576 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2023-10-06T04:17:13.577 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:17:13.577 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2023-10-06T04:17:13.635 INFO:teuthology.orchestra.run.smithi177.stdout:4.18.0-489.el8.x86_64 2023-10-06T04:17:13.636 INFO:teuthology.task.kernel:Running kernel on smithi177: 4.18.0-489.el8.x86_64 2023-10-06T04:17:13.636 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum install -y kernel 2023-10-06T04:17:13.639 INFO:teuthology.orchestra.run.smithi123.stdout:4.18.0-489.el8.x86_64 2023-10-06T04:17:13.639 INFO:teuthology.task.kernel:Running kernel on smithi123: 4.18.0-489.el8.x86_64 2023-10-06T04:17:13.639 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum install -y kernel 2023-10-06T04:17:19.981 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - AppStream 8.8 MB/s | 33 MB 00:03 2023-10-06T04:17:20.347 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - AppStream 8.0 MB/s | 33 MB 00:04 2023-10-06T04:17:30.446 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - BaseOS 12 MB/s | 48 MB 00:04 2023-10-06T04:17:30.689 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - BaseOS 10 MB/s | 48 MB 00:04 2023-10-06T04:17:37.449 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - Extras 118 kB/s | 18 kB 00:00 2023-10-06T04:17:37.697 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - Extras 105 kB/s | 18 kB 00:00 2023-10-06T04:17:38.022 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - Extras common packages 17 kB/s | 6.9 kB 00:00 2023-10-06T04:17:38.097 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - Extras common packages 31 kB/s | 6.9 kB 00:00 2023-10-06T04:17:38.324 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 23 kB/s | 3.5 kB 00:00 2023-10-06T04:17:38.390 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 24 kB/s | 3.5 kB 00:00 2023-10-06T04:17:39.107 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 25 MB/s | 16 MB 00:00 2023-10-06T04:17:39.497 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 17 MB/s | 16 MB 00:00 2023-10-06T04:17:42.899 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 300 kB/s | 24 kB 00:00 2023-10-06T04:17:43.383 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 368 kB/s | 24 kB 00:00 2023-10-06T04:17:46.415 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-06T04:17:46.415 INFO:teuthology.orchestra.run.smithi177.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-06T04:17:46.660 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-10-06T04:17:46.660 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout: Package Architecture Version Repository Size 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout:================================================================================ 2023-10-06T04:17:46.661 INFO:teuthology.orchestra.run.smithi177.stdout:Install 3 Packages 2023-10-06T04:17:46.662 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:17:46.662 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 88 M 2023-10-06T04:17:46.662 INFO:teuthology.orchestra.run.smithi177.stdout:Installed size: 96 M 2023-10-06T04:17:46.662 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2023-10-06T04:17:46.857 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-4.18.0-348.el8.x86_64 is already installed. 2023-10-06T04:17:46.857 INFO:teuthology.orchestra.run.smithi123.stdout:Package kernel-4.18.0-489.el8.x86_64 is already installed. 2023-10-06T04:17:47.113 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-10-06T04:17:47.114 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-10-06T04:17:47.114 INFO:teuthology.orchestra.run.smithi123.stdout: Package Architecture Version Repository Size 2023-10-06T04:17:47.114 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-10-06T04:17:47.114 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2023-10-06T04:17:47.114 INFO:teuthology.orchestra.run.smithi123.stdout: kernel x86_64 4.18.0-514.el8 baseos 10 M 2023-10-06T04:17:47.114 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-10-06T04:17:47.114 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-core x86_64 4.18.0-514.el8 baseos 43 M 2023-10-06T04:17:47.115 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules x86_64 4.18.0-514.el8 baseos 35 M 2023-10-06T04:17:47.115 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:17:47.115 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-10-06T04:17:47.115 INFO:teuthology.orchestra.run.smithi123.stdout:================================================================================ 2023-10-06T04:17:47.115 INFO:teuthology.orchestra.run.smithi123.stdout:Install 3 Packages 2023-10-06T04:17:47.115 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:17:47.115 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 88 M 2023-10-06T04:17:47.116 INFO:teuthology.orchestra.run.smithi123.stdout:Installed size: 96 M 2023-10-06T04:17:47.116 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-10-06T04:17:50.108 INFO:teuthology.orchestra.run.smithi177.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 3.1 MB/s | 10 MB 00:03 2023-10-06T04:17:50.442 INFO:teuthology.orchestra.run.smithi123.stdout:(1/3): kernel-4.18.0-514.el8.x86_64.rpm 3.2 MB/s | 10 MB 00:03 2023-10-06T04:17:52.017 INFO:teuthology.orchestra.run.smithi177.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 6.9 MB/s | 35 MB 00:05 2023-10-06T04:17:52.450 INFO:teuthology.orchestra.run.smithi123.stdout:(2/3): kernel-modules-4.18.0-514.el8.x86_64.rpm 7.0 MB/s | 35 MB 00:05 2023-10-06T04:17:53.467 INFO:teuthology.orchestra.run.smithi123.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 7.0 MB/s | 43 MB 00:06 2023-10-06T04:17:53.468 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:17:53.468 INFO:teuthology.orchestra.run.smithi123.stdout:Total 14 MB/s | 88 MB 00:06 2023-10-06T04:17:54.137 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-10-06T04:17:54.325 INFO:teuthology.orchestra.run.smithi177.stdout:(3/3): kernel-core-4.18.0-514.el8.x86_64.rpm 5.7 MB/s | 43 MB 00:07 2023-10-06T04:17:54.325 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:17:54.325 INFO:teuthology.orchestra.run.smithi177.stdout:Total 11 MB/s | 88 MB 00:07 2023-10-06T04:17:54.548 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-10-06T04:17:54.548 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-10-06T04:17:54.996 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2023-10-06T04:17:55.406 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2023-10-06T04:17:55.407 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2023-10-06T04:17:55.789 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-10-06T04:17:55.912 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-10-06T04:17:56.639 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2023-10-06T04:17:56.667 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2023-10-06T04:17:58.698 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-10-06T04:17:59.224 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:17:59.462 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2023-10-06T04:18:00.037 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:18:02.798 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:18:03.206 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:18:03.554 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:18:03.953 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:18:06.583 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:18:06.878 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:07.331 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:18:07.666 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:28.543 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:28.568 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:28.913 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:28.913 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:18:28.913 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:18:29.350 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-core-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:29.373 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:29.658 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:29.658 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:18:29.658 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-10-06T04:18:29.659 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-06T04:18:29.659 INFO:teuthology.orchestra.run.smithi123.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-06T04:18:29.659 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:18:29.659 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-10-06T04:18:29.719 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: kernel-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-4.18.0-514.el8.x86_64 1/3 2023-10-06T04:18:29.720 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-core-4.18.0-514.el8.x86_64 2/3 2023-10-06T04:18:29.800 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:18:30.018 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T04:18:30.018 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T04:18:30.019 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-06T04:18:30.019 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T04:18:30.019 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T04:18:30.019 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2023-10-06T04:18:30.019 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi123.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T04:18:30.019 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:18:30.227 INFO:teuthology.orchestra.run.smithi123.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T04:18:30.227 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T04:18:30.227 DEBUG:teuthology.orchestra.run.smithi123:> sudo rpm -qi grub2-tools 2023-10-06T04:18:30.261 INFO:teuthology.orchestra.run.smithi123.stdout:Name : grub2-tools 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Epoch : 1 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Version : 2.02 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Release : 129.el8 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Architecture: x86_64 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Group : System Environment/Base 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Size : 9265586 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:License : GPLv3+ 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-06T04:18:30.262 INFO:teuthology.orchestra.run.smithi123.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:Relocations : (not relocatable) 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:Packager : CentOS BuildSystem 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:Vendor : CentOS 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:Summary : Support tools for GRUB. 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:Description : 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T04:18:30.263 INFO:teuthology.orchestra.run.smithi123.stdout:hardware devices. 2023-10-06T04:18:30.264 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:18:30.264 INFO:teuthology.orchestra.run.smithi123.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T04:18:30.264 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T04:18:30.265 INFO:teuthology.task.kernel:Updating grub on smithi123 to boot 4.18.0-514.el8.x86_64 2023-10-06T04:18:30.265 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T04:18:30.452 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : kernel-modules-4.18.0-514.el8.x86_64 3/3 2023-10-06T04:18:30.452 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:18:30.452 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2023-10-06T04:18:30.452 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-4.18.0-514.el8.x86_64 kernel-core-4.18.0-514.el8.x86_64 2023-10-06T04:18:30.453 INFO:teuthology.orchestra.run.smithi177.stdout: kernel-modules-4.18.0-514.el8.x86_64 2023-10-06T04:18:30.453 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:18:30.453 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-10-06T04:18:30.629 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:18:30.847 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T04:18:30.848 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T04:18:30.848 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 4.18.0-489.el8.x86_64 Expected: 4.18.0-514.el8.x86_64 2023-10-06T04:18:30.848 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2023-10-06T04:18:30.848 DEBUG:teuthology.task.kernel:src is distro, skipping download 2023-10-06T04:18:30.848 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2023-10-06T04:18:30.848 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi177.front.sepia.ceph.com, path=None, version=distro) 2023-10-06T04:18:30.848 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q kernel | sort -rV | head -n 1 2023-10-06T04:18:31.057 INFO:teuthology.orchestra.run.smithi177.stdout:kernel-4.18.0-514.el8.x86_64 2023-10-06T04:18:31.058 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 4.18.0-514.el8.x86_64 2023-10-06T04:18:31.058 DEBUG:teuthology.orchestra.run.smithi177:> sudo rpm -qi grub2-tools 2023-10-06T04:18:31.091 INFO:teuthology.orchestra.run.smithi177.stdout:Name : grub2-tools 2023-10-06T04:18:31.091 INFO:teuthology.orchestra.run.smithi177.stdout:Epoch : 1 2023-10-06T04:18:31.091 INFO:teuthology.orchestra.run.smithi177.stdout:Version : 2.02 2023-10-06T04:18:31.091 INFO:teuthology.orchestra.run.smithi177.stdout:Release : 129.el8 2023-10-06T04:18:31.091 INFO:teuthology.orchestra.run.smithi177.stdout:Architecture: x86_64 2023-10-06T04:18:31.091 INFO:teuthology.orchestra.run.smithi177.stdout:Install Date: Thu 04 May 2023 10:17:48 PM UTC 2023-10-06T04:18:31.091 INFO:teuthology.orchestra.run.smithi177.stdout:Group : System Environment/Base 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:Size : 9265586 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:License : GPLv3+ 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:Signature : RSA/SHA256, Tue 19 Jul 2022 02:52:06 PM UTC, Key ID 05b555b38483c65d 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:Source RPM : grub2-2.02-129.el8.src.rpm 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:Build Date : Tue 19 Jul 2022 02:30:39 PM UTC 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:Build Host : kbuilder.bsys.centos.org 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:Relocations : (not relocatable) 2023-10-06T04:18:31.092 INFO:teuthology.orchestra.run.smithi177.stdout:Packager : CentOS BuildSystem 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:Vendor : CentOS 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:URL : http://www.gnu.org/software/grub/ 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:Summary : Support tools for GRUB. 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:Description : 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:customizable bootloader with modular architecture. It supports a rich 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:variety of kernel formats, file systems, computer architectures and 2023-10-06T04:18:31.093 INFO:teuthology.orchestra.run.smithi177.stdout:hardware devices. 2023-10-06T04:18:31.094 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:18:31.094 INFO:teuthology.orchestra.run.smithi177.stdout:This subpackage provides tools for support of all platforms. 2023-10-06T04:18:31.095 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2023-10-06T04:18:31.095 INFO:teuthology.task.kernel:Updating grub on smithi177 to boot 4.18.0-514.el8.x86_64 2023-10-06T04:18:31.095 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2023-10-06T04:18:31.980 INFO:teuthology.orchestra.run.smithi123.stderr:Generating grub configuration file ... 2023-10-06T04:18:32.691 INFO:teuthology.orchestra.run.smithi177.stderr:Generating grub configuration file ... 2023-10-06T04:18:34.432 INFO:teuthology.orchestra.run.smithi123.stderr:done 2023-10-06T04:18:34.434 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2023-10-06T04:18:34.512 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.XQkzS5Nh3O 2023-10-06T04:18:34.512 DEBUG:teuthology.orchestra.run.smithi123:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.XQkzS5Nh3O 2023-10-06T04:18:34.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.XQkzS5Nh3O 2023-10-06T04:18:34.724 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.XQkzS5Nh3O is 6KB 2023-10-06T04:18:34.780 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.XQkzS5Nh3O 2023-10-06T04:18:34.805 DEBUG:teuthology.orchestra.run.smithi123:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T04:18:34.889 INFO:teuthology.orchestra.run.smithi123.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-06T04:18:34.889 INFO:teuthology.orchestra.run.smithi123.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-06T04:18:34.889 INFO:teuthology.orchestra.run.smithi123.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-06T04:18:34.889 INFO:teuthology.orchestra.run.smithi123.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-06T04:18:34.890 DEBUG:teuthology.orchestra.run.smithi123:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-06T04:18:34.984 INFO:teuthology.orchestra.run.smithi177.stderr:done 2023-10-06T04:18:34.985 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-10-06T04:18:35.014 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.f1AC1f9V1c 2023-10-06T04:18:35.015 DEBUG:teuthology.orchestra.run.smithi177:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.f1AC1f9V1c 2023-10-06T04:18:35.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo shutdown -r now 2023-10-06T04:18:35.094 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.f1AC1f9V1c 2023-10-06T04:18:35.243 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.f1AC1f9V1c is 6KB 2023-10-06T04:18:35.296 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.f1AC1f9V1c 2023-10-06T04:18:35.314 DEBUG:teuthology.orchestra.run.smithi177:> sudo /bin/ls /boot/loader/entries || true 2023-10-06T04:18:35.401 INFO:teuthology.orchestra.run.smithi177.stdout:a1b29d6c289747bdb22930598d39874a-0-rescue.conf 2023-10-06T04:18:35.401 INFO:teuthology.orchestra.run.smithi177.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-348.el8.x86_64.conf 2023-10-06T04:18:35.401 INFO:teuthology.orchestra.run.smithi177.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-489.el8.x86_64.conf 2023-10-06T04:18:35.401 INFO:teuthology.orchestra.run.smithi177.stdout:a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64.conf 2023-10-06T04:18:35.402 DEBUG:teuthology.orchestra.run.smithi177:> sudo grub2-set-default a1b29d6c289747bdb22930598d39874a-4.18.0-514.el8.x86_64 2023-10-06T04:18:35.536 DEBUG:teuthology.orchestra.run.smithi177:> sudo shutdown -r now 2023-10-06T04:19:05.074 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2023-10-06T04:19:05.075 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-06T04:19:05.075 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:19:05.539 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2023-10-06T04:19:05.540 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T04:19:05.540 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:19:23.642 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-10-06T04:19:23.902 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-10-06T04:19:32.650 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-06T04:19:32.651 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:19:32.903 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T04:19:32.904 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:19:35.706 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-10-06T04:19:35.966 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-10-06T04:19:47.718 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-06T04:19:47.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:19:47.967 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T04:19:47.968 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:20:06.142 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.123 2023-10-06T04:20:06.398 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.177 2023-10-06T04:20:21.158 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-06T04:20:21.159 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:20:21.399 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T04:20:21.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:20:21.494 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-10-06T04:20:21.977 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-06T04:20:21.977 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "4.18.0-514.el8.x86_64"... 2023-10-06T04:20:21.977 DEBUG:teuthology.orchestra.run.smithi123:> uname -r 2023-10-06T04:20:21.995 INFO:teuthology.orchestra.run.smithi123.stdout:4.18.0-514.el8.x86_64 2023-10-06T04:20:21.995 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-06T04:20:21.995 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T04:20:21.996 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T04:20:22.888 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-10-06T04:20:22.997 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2023-10-06T04:20:22.997 DEBUG:teuthology.orchestra.run.smithi123:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T04:20:23.471 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T04:20:23.471 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "4.18.0-514.el8.x86_64"... 2023-10-06T04:20:23.471 DEBUG:teuthology.orchestra.run.smithi177:> uname -r 2023-10-06T04:20:23.488 INFO:teuthology.orchestra.run.smithi177.stdout:4.18.0-514.el8.x86_64 2023-10-06T04:20:23.488 DEBUG:teuthology.task.kernel:current kernel version is 4.18.0-514.el8.x86_64 vs 4.18.0-514.el8.x86_64 2023-10-06T04:20:23.489 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-10-06T04:20:23.489 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2023-10-06T04:20:23.730 INFO:teuthology.orchestra.run.smithi123.stdout:ttyS1 2023-10-06T04:20:23.756 DEBUG:teuthology.parallel:result is None 2023-10-06T04:20:24.489 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2023-10-06T04:20:24.511 DEBUG:teuthology.orchestra.run.smithi177:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2023-10-06T04:20:24.579 INFO:teuthology.orchestra.run.smithi177.stdout:ttyS1 2023-10-06T04:20:24.591 DEBUG:teuthology.parallel:result is None 2023-10-06T04:20:24.592 INFO:teuthology.run_tasks:Running task internal.base... 2023-10-06T04:20:24.598 INFO:teuthology.task.internal:Creating test directory... 2023-10-06T04:20:24.598 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T04:20:24.600 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-10-06T04:20:24.620 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-10-06T04:20:24.752 INFO:teuthology.run_tasks:Running task internal.archive... 2023-10-06T04:20:24.788 INFO:teuthology.task.internal:Creating archive directory... 2023-10-06T04:20:24.788 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T04:20:24.790 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-10-06T04:20:24.830 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-10-06T04:20:24.846 INFO:teuthology.task.internal:Enabling coredump saving... 2023-10-06T04:20:24.846 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T04:20:24.870 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-10-06T04:20:24.904 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:20:24.949 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:20:24.950 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:20:24.950 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-10-06T04:20:24.976 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-10-06T04:20:24.983 INFO:teuthology.task.internal:Configuring sudo... 2023-10-06T04:20:24.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T04:20:24.992 DEBUG:teuthology.orchestra.run.smithi177:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-10-06T04:20:25.022 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-10-06T04:20:25.032 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-10-06T04:20:25.032 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T04:20:25.060 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-10-06T04:20:25.082 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:20:25.159 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:20:25.216 DEBUG:teuthology.orchestra.run.smithi123:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:20:25.305 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:20:25.343 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:20:25.343 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T04:20:25.410 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:20:25.455 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2023-10-06T04:20:25.513 DEBUG:teuthology.orchestra.run.smithi177:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:20:25.601 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2023-10-06T04:20:25.645 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:20:25.645 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-10-06T04:20:25.713 DEBUG:teuthology.orchestra.run.smithi123:> sudo service rsyslog restart 2023-10-06T04:20:25.716 DEBUG:teuthology.orchestra.run.smithi177:> sudo service rsyslog restart 2023-10-06T04:20:25.775 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:20:25.819 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:20:26.259 INFO:teuthology.run_tasks:Running task internal.timer... 2023-10-06T04:20:26.267 INFO:teuthology.task.internal:Starting timer... 2023-10-06T04:20:26.267 INFO:teuthology.run_tasks:Running task pcp... 2023-10-06T04:20:26.313 INFO:teuthology.run_tasks:Running task selinux... 2023-10-06T04:20:26.358 DEBUG:teuthology.task:Applying overrides for task selinux: {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']} 2023-10-06T04:20:26.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo service auditd rotate 2023-10-06T04:20:26.431 INFO:teuthology.orchestra.run.smithi123.stdout:Rotating logs: 2023-10-06T04:20:26.433 DEBUG:teuthology.orchestra.run.smithi177:> sudo service auditd rotate 2023-10-06T04:20:26.525 INFO:teuthology.orchestra.run.smithi177.stdout:Rotating logs: 2023-10-06T04:20:26.527 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-10-06T04:20:26.527 DEBUG:teuthology.orchestra.run.smithi123:> /usr/sbin/getenforce 2023-10-06T04:20:26.548 INFO:teuthology.orchestra.run.smithi123.stdout:Permissive 2023-10-06T04:20:26.548 DEBUG:teuthology.orchestra.run.smithi177:> /usr/sbin/getenforce 2023-10-06T04:20:26.574 INFO:teuthology.orchestra.run.smithi177.stdout:Permissive 2023-10-06T04:20:26.574 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi123.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi177.front.sepia.ceph.com': 'permissive'} 2023-10-06T04:20:26.574 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:20:26.615 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:20:26.616 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:20:26.643 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:20:26.644 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-10-06T04:20:26.644 DEBUG:teuthology.orchestra.run.smithi123:> sudo /usr/sbin/setenforce permissive 2023-10-06T04:20:26.684 DEBUG:teuthology.orchestra.run.smithi177:> sudo /usr/sbin/setenforce permissive 2023-10-06T04:20:26.711 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-10-06T04:20:26.722 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-10-06T04:20:26.722 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-10-06T04:20:26.743 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-10-06T04:20:26.744 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi123.front.sepia.ceph.com,smithi177.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-10-06T04:27:51.216 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi123.front.sepia.ceph.com'), Remote(name='ubuntu@smithi177.front.sepia.ceph.com')] 2023-10-06T04:27:51.218 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-06T04:27:51.218 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi123.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:27:51.284 DEBUG:teuthology.orchestra.run.smithi123:> true 2023-10-06T04:27:51.353 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi123.front.sepia.ceph.com' 2023-10-06T04:27:51.353 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T04:27:51.354 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi177.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-10-06T04:27:51.425 DEBUG:teuthology.orchestra.run.smithi177:> true 2023-10-06T04:27:51.498 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi177.front.sepia.ceph.com' 2023-10-06T04:27:51.498 INFO:teuthology.run_tasks:Running task clock... 2023-10-06T04:27:51.510 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-10-06T04:27:51.510 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T04:27:51.511 DEBUG:teuthology.orchestra.run.smithi123:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:27:51.513 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-10-06T04:27:51.514 DEBUG:teuthology.orchestra.run.smithi177:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:27:51.543 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T04:27:51.557 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T04:27:51.571 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2023-10-06T04:27:51.585 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2023-10-06T04:27:51.620 INFO:teuthology.orchestra.run.smithi123.stderr:sudo: ntpd: command not found 2023-10-06T04:27:51.631 INFO:teuthology.orchestra.run.smithi123.stdout:506 Cannot talk to daemon 2023-10-06T04:27:51.645 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T04:27:51.648 INFO:teuthology.orchestra.run.smithi177.stderr:sudo: ntpd: command not found 2023-10-06T04:27:51.658 INFO:teuthology.orchestra.run.smithi123.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T04:27:51.659 INFO:teuthology.orchestra.run.smithi177.stdout:506 Cannot talk to daemon 2023-10-06T04:27:51.673 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntp.service: Unit ntp.service not found. 2023-10-06T04:27:51.686 INFO:teuthology.orchestra.run.smithi177.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2023-10-06T04:27:51.724 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-10-06T04:27:51.727 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:27:51.727 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-10-06T04:27:51.727 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.727 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.727 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.727 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.755 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-10-06T04:27:51.759 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:27:51.760 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-10-06T04:27:51.760 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.760 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.760 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.760 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:27:51.760 INFO:teuthology.run_tasks:Running task install... 2023-10-06T04:27:51.769 DEBUG:teuthology.task.install:project ceph 2023-10-06T04:27:51.770 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}} 2023-10-06T04:27:51.770 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'} 2023-10-06T04:27:51.770 INFO:teuthology.task.install:Using flavor: default 2023-10-06T04:27:51.775 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-10-06T04:27:51.776 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']} 2023-10-06T04:27:51.776 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T04:27:51.776 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T04:27:51.778 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'tag': None, 'wait_for_package': False} 2023-10-06T04:27:51.778 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T04:27:52.013 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T04:27:52.013 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T04:27:52.021 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/ 2023-10-06T04:27:52.021 INFO:teuthology.task.install.rpm:Package version is 17.2.6-1296.g736ac5ba 2023-10-06T04:27:52.183 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T04:27:52.183 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:27:52.183 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T04:27:52.188 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-yuri-testing-2023-10-05-0730-quincy/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/centos/8/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2023-10-06T04:27:52.189 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:27:52.189 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/yum.repos.d/ceph.repo 2023-10-06T04:27:52.214 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T04:27:52.215 DEBUG:teuthology.orchestra.run.smithi123:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T04:27:52.216 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel on remote rpm x86_64 2023-10-06T04:27:52.217 DEBUG:teuthology.orchestra.run.smithi177:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/736ac5ba72cb47af7d6f6666b1e100f7e20a20e2/;g' /etc/yum.repos.d/ceph.repo ; fi 2023-10-06T04:27:52.285 DEBUG:teuthology.orchestra.run.smithi177:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T04:27:52.286 DEBUG:teuthology.orchestra.run.smithi123:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2023-10-06T04:27:52.363 DEBUG:teuthology.orchestra.run.smithi177:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T04:27:52.365 DEBUG:teuthology.orchestra.run.smithi123:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2023-10-06T04:27:52.429 INFO:teuthology.orchestra.run.smithi177.stdout:check_obsoletes = 1 2023-10-06T04:27:52.431 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum clean all 2023-10-06T04:27:52.432 INFO:teuthology.orchestra.run.smithi123.stdout:check_obsoletes = 1 2023-10-06T04:27:52.433 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum clean all 2023-10-06T04:27:52.705 INFO:teuthology.orchestra.run.smithi123.stdout:76 files removed 2023-10-06T04:27:52.720 INFO:teuthology.orchestra.run.smithi177.stdout:76 files removed 2023-10-06T04:27:52.735 DEBUG:teuthology.orchestra.run.smithi123:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T04:27:52.748 DEBUG:teuthology.orchestra.run.smithi177:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel 2023-10-06T04:27:53.451 INFO:teuthology.orchestra.run.smithi177.stdout:ceph packages for x86_64 264 kB/s | 78 kB 00:00 2023-10-06T04:27:53.498 INFO:teuthology.orchestra.run.smithi123.stdout:ceph packages for x86_64 218 kB/s | 78 kB 00:00 2023-10-06T04:27:53.826 INFO:teuthology.orchestra.run.smithi177.stdout:ceph noarch packages 27 kB/s | 9.2 kB 00:00 2023-10-06T04:27:53.866 INFO:teuthology.orchestra.run.smithi123.stdout:ceph noarch packages 28 kB/s | 9.2 kB 00:00 2023-10-06T04:27:54.202 INFO:teuthology.orchestra.run.smithi177.stdout:ceph source packages 4.2 kB/s | 1.4 kB 00:00 2023-10-06T04:27:54.256 INFO:teuthology.orchestra.run.smithi123.stdout:ceph source packages 4.0 kB/s | 1.4 kB 00:00 2023-10-06T04:27:55.533 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - AppStream 27 MB/s | 33 MB 00:01 2023-10-06T04:27:55.819 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - AppStream 21 MB/s | 33 MB 00:01 2023-10-06T04:28:07.174 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - BaseOS 8.6 MB/s | 48 MB 00:05 2023-10-06T04:28:10.628 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - BaseOS 5.5 MB/s | 48 MB 00:08 2023-10-06T04:28:14.399 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - Extras 55 kB/s | 18 kB 00:00 2023-10-06T04:28:14.909 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - Extras common packages 21 kB/s | 6.9 kB 00:00 2023-10-06T04:28:15.943 INFO:teuthology.orchestra.run.smithi123.stdout:CentOS Stream 8 - PowerTools 7.3 MB/s | 6.5 MB 00:00 2023-10-06T04:28:17.370 INFO:teuthology.orchestra.run.smithi123.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-06T04:28:17.836 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - Extras 56 kB/s | 18 kB 00:00 2023-10-06T04:28:18.361 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - Extras common packages 20 kB/s | 6.9 kB 00:00 2023-10-06T04:28:19.262 INFO:teuthology.orchestra.run.smithi123.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-10-06T04:28:19.504 INFO:teuthology.orchestra.run.smithi177.stdout:CentOS Stream 8 - PowerTools 6.5 MB/s | 6.5 MB 00:00 2023-10-06T04:28:20.930 INFO:teuthology.orchestra.run.smithi177.stdout:Copr repo for python3-asyncssh owned by ceph 25 kB/s | 3.5 kB 00:00 2023-10-06T04:28:22.839 INFO:teuthology.orchestra.run.smithi177.stdout:Extra Packages for Enterprise Linux 9.1 MB/s | 16 MB 00:01 2023-10-06T04:28:23.129 INFO:teuthology.orchestra.run.smithi123.stdout:lab-extras 365 kB/s | 24 kB 00:00 2023-10-06T04:28:26.656 INFO:teuthology.orchestra.run.smithi177.stdout:lab-extras 399 kB/s | 24 kB 00:00 2023-10-06T04:28:27.410 INFO:teuthology.orchestra.run.smithi123.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:28:27.411 INFO:teuthology.orchestra.run.smithi123.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:28:27.425 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-06T04:28:27.425 INFO:teuthology.orchestra.run.smithi123.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-06T04:28:27.426 INFO:teuthology.orchestra.run.smithi123.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-06T04:28:27.427 INFO:teuthology.orchestra.run.smithi123.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-06T04:28:27.428 INFO:teuthology.orchestra.run.smithi123.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T04:28:27.430 INFO:teuthology.orchestra.run.smithi123.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T04:28:27.433 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T04:28:27.434 INFO:teuthology.orchestra.run.smithi123.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T04:28:27.646 INFO:teuthology.orchestra.run.smithi123.stdout:Dependencies resolved. 2023-10-06T04:28:27.659 INFO:teuthology.orchestra.run.smithi123.stdout:==================================================================================================== 2023-10-06T04:28:27.659 INFO:teuthology.orchestra.run.smithi123.stdout: Package Arch Version Repository Size 2023-10-06T04:28:27.659 INFO:teuthology.orchestra.run.smithi123.stdout:==================================================================================================== 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout:Installing: 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T04:28:27.660 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T04:28:27.661 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T04:28:27.661 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T04:28:27.661 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T04:28:27.661 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T04:28:27.661 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T04:28:27.661 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T04:28:27.661 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T04:28:27.662 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T04:28:27.662 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T04:28:27.662 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-06T04:28:27.662 INFO:teuthology.orchestra.run.smithi123.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-06T04:28:27.662 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-06T04:28:27.662 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T04:28:27.663 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrading: 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-06T04:28:27.664 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-06T04:28:27.665 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-06T04:28:27.666 INFO:teuthology.orchestra.run.smithi123.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-06T04:28:27.666 INFO:teuthology.orchestra.run.smithi123.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-06T04:28:27.666 INFO:teuthology.orchestra.run.smithi123.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-06T04:28:27.666 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-06T04:28:27.666 INFO:teuthology.orchestra.run.smithi123.stdout:Installing dependencies: 2023-10-06T04:28:27.666 INFO:teuthology.orchestra.run.smithi123.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-06T04:28:27.667 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T04:28:27.667 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T04:28:27.667 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T04:28:27.667 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T04:28:27.667 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T04:28:27.667 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T04:28:27.668 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T04:28:27.668 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T04:28:27.668 INFO:teuthology.orchestra.run.smithi123.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-06T04:28:27.668 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T04:28:27.668 INFO:teuthology.orchestra.run.smithi123.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-06T04:28:27.668 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-06T04:28:27.668 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-06T04:28:27.669 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T04:28:27.669 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-06T04:28:27.669 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-06T04:28:27.669 INFO:teuthology.orchestra.run.smithi123.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T04:28:27.669 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-06T04:28:27.670 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-06T04:28:27.670 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T04:28:27.670 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-06T04:28:27.670 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T04:28:27.670 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-06T04:28:27.671 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-06T04:28:27.671 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T04:28:27.671 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-06T04:28:27.671 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-06T04:28:27.671 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-06T04:28:27.671 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T04:28:27.672 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-06T04:28:27.673 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T04:28:27.674 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T04:28:27.675 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-06T04:28:27.676 INFO:teuthology.orchestra.run.smithi123.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction Summary 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout:==================================================================================================== 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout:Install 123 Packages 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout:Upgrade 16 Packages 2023-10-06T04:28:27.677 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:28:27.678 INFO:teuthology.orchestra.run.smithi123.stdout:Total download size: 210 M 2023-10-06T04:28:27.678 INFO:teuthology.orchestra.run.smithi123.stdout:Downloading Packages: 2023-10-06T04:28:28.171 INFO:teuthology.orchestra.run.smithi123.stdout:(1/139): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 50 kB/s | 6.4 kB 00:00 2023-10-06T04:28:28.346 INFO:teuthology.orchestra.run.smithi123.stdout:(2/139): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 4.9 MB/s | 876 kB 00:00 2023-10-06T04:28:28.430 INFO:teuthology.orchestra.run.smithi123.stdout:(3/139): ceph-immutable-object-cache-17.2.6-129 1.9 MB/s | 162 kB 00:00 2023-10-06T04:28:28.701 INFO:teuthology.orchestra.run.smithi123.stdout:(4/139): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 8.5 MB/s | 2.3 MB 00:00 2023-10-06T04:28:28.889 INFO:teuthology.orchestra.run.smithi123.stdout:(5/139): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 8.4 MB/s | 1.6 MB 00:00 2023-10-06T04:28:29.006 INFO:teuthology.orchestra.run.smithi123.stdout:(6/139): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 5.4 MB/s | 5.2 MB 00:00 2023-10-06T04:28:29.340 INFO:teuthology.orchestra.run.smithi123.stdout:(7/139): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 8.7 MB/s | 3.9 MB 00:00 2023-10-06T04:28:30.715 INFO:teuthology.orchestra.run.smithi123.stdout:(8/139): ceph-osd-17.2.6-1296.g736ac5ba.el8.x86 9.8 MB/s | 17 MB 00:01 2023-10-06T04:28:30.815 INFO:teuthology.orchestra.run.smithi123.stdout:(9/139): ceph-selinux-17.2.6-1296.g736ac5ba.el8 244 kB/s | 24 kB 00:00 2023-10-06T04:28:30.850 INFO:teuthology.orchestra.run.smithi177.stdout:Package librados2-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:28:30.851 INFO:teuthology.orchestra.run.smithi177.stdout:Package librbd1-1:12.2.7-9.el8.x86_64 is already installed. 2023-10-06T04:28:30.864 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-1:1.40.16-4.el8.x86_64 is already installed. 2023-10-06T04:28:30.864 INFO:teuthology.orchestra.run.smithi177.stdout:Package NetworkManager-initscripts-updown-1:1.40.16-4.el8.noarch is already installed. 2023-10-06T04:28:30.865 INFO:teuthology.orchestra.run.smithi177.stdout:Package iproute-6.2.0-1.el8.x86_64 is already installed. 2023-10-06T04:28:30.866 INFO:teuthology.orchestra.run.smithi177.stdout:Package util-linux-2.32.1-41.el8.x86_64 is already installed. 2023-10-06T04:28:30.867 INFO:teuthology.orchestra.run.smithi177.stdout:Package libacl-devel-2.2.53-1.el8.x86_64 is already installed. 2023-10-06T04:28:30.869 INFO:teuthology.orchestra.run.smithi177.stdout:Package libattr-devel-2.4.48-3.el8.x86_64 is already installed. 2023-10-06T04:28:30.872 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsdump-3.1.8-4.el8.x86_64 is already installed. 2023-10-06T04:28:30.873 INFO:teuthology.orchestra.run.smithi177.stdout:Package xfsprogs-5.0.0-10.el8.x86_64 is already installed. 2023-10-06T04:28:31.086 INFO:teuthology.orchestra.run.smithi177.stdout:Dependencies resolved. 2023-10-06T04:28:31.099 INFO:teuthology.orchestra.run.smithi177.stdout:==================================================================================================== 2023-10-06T04:28:31.099 INFO:teuthology.orchestra.run.smithi177.stdout: Package Arch Version Repository Size 2023-10-06T04:28:31.099 INFO:teuthology.orchestra.run.smithi177.stdout:==================================================================================================== 2023-10-06T04:28:31.099 INFO:teuthology.orchestra.run.smithi177.stdout:Installing: 2023-10-06T04:28:31.099 INFO:teuthology.orchestra.run.smithi177.stdout: bison x86_64 3.0.4-10.el8 appstream 688 k 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 6.4 k 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.2 M 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 876 k 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 162 k 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 1.6 M 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 121 k 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 1.7 M 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 7.4 M 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 47 k 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T04:28:31.100 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 52 M 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 267 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 80 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 255 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 18 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel x86_64 0.189-3.el8 baseos 62 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: flex x86_64 2.6.1-9.el8 appstream 320 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel x86_64 0.3.112-1.el8 baseos 19 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 25 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 797 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 130 k 2023-10-06T04:28:31.101 INFO:teuthology.orchestra.run.smithi177.stdout: libtool x86_64 2.4.6-25.el8 appstream 709 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel x86_64 2.32.1-43.el8 baseos 100 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel x86_64 1:1.1.1k-9.el8 baseos 3.2 M 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 190 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 371 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 358 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 107 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 90 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 173 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel x86_64 5.0.0-12.el8 baseos 74 k 2023-10-06T04:28:31.102 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrading: 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown noarch 1:1.40.16-9.el8 baseos 143 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client x86_64 0.189-3.el8 baseos 76 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf x86_64 0.189-3.el8 baseos 232 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs x86_64 0.189-3.el8 baseos 303 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: iproute x86_64 6.2.0-5.el8 baseos 854 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid x86_64 2.32.1-43.el8 baseos 220 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk x86_64 2.32.1-43.el8 baseos 254 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: libmount x86_64 2.32.1-43.el8 baseos 236 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: librados2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.5 M 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.6 M 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols x86_64 2.32.1-43.el8 baseos 179 k 2023-10-06T04:28:31.103 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid x86_64 2.32.1-43.el8 baseos 99 k 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux x86_64 2.32.1-43.el8 baseos 2.5 M 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user x86_64 2.32.1-43.el8 baseos 103 k 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump x86_64 3.1.8-6.el8 baseos 334 k 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs x86_64 5.0.0-12.el8 baseos 1.1 M 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout:Installing dependencies: 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: automake noarch 1.16.1-8.el8 appstream 740 k 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 23 M 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 23 k 2023-10-06T04:28:31.104 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 2.3 M 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 238 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 3.9 M 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 17 M 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts noarch 2:17.2.6-1296.g736ac5ba.el8 ceph-noarch 14 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 24 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: fuse x86_64 2.9.7-17.el8 baseos 83 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs x86_64 1:2.7-9.el8 epel 306 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: jq x86_64 1.6-7.el8 appstream 202 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel x86_64 1.5.10-9.el8 baseos 48 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel x86_64 1.18.2-22.el8 baseos 561 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 174 k 2023-10-06T04:28:31.105 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel x86_64 1.45.6-5.el8 baseos 39 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran x86_64 8.5.0-20.el8 baseos 645 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: liboath x86_64 2.6.2-3.el8 epel 59 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath x86_64 8.5.0-20.el8 baseos 172 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq x86_64 0.9.0-4.el8 baseos 47 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 507 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka x86_64 0.11.4-3.el8 appstream 354 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2 x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 5.6 M 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel x86_64 2.9-8.el8 baseos 200 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel x86_64 2.9-3.el8 baseos 87 k 2023-10-06T04:28:31.106 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind x86_64 1.3.1-3.el8 epel 75 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel x86_64 0.3.2-2.el8 baseos 18 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel x86_64 1.4.4-1.el8 baseos 44 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust x86_64 2.8.1-11.el8 appstream 259 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma x86_64 6.8.2-2.el8 appstream 187 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: openblas x86_64 0.3.15-4.el8 appstream 4.7 M 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads x86_64 0.3.15-4.el8 appstream 4.9 M 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel x86_64 10.32-3.el8 baseos 605 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16 x86_64 10.32-3.el8 baseos 229 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32 x86_64 10.32-3.el8 baseos 220 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh noarch 2.7.0-2.el8 epel 476 k 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel noarch 2.5.1-7.el8 appstream 4.8 M 2023-10-06T04:28:31.107 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt x86_64 3.1.6-2.el8.1 epel 44 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4 noarch 4.6.3-2.el8.1 epel 185 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools noarch 3.1.1-4.el8 epel 33 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 45 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common x86_64 2:17.2.6-1296.g736ac5ba.el8 ceph 115 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi noarch 2018.10.15-7.el8 epel 16 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi x86_64 1.11.5-6.el8 baseos 249 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot noarch 8.5.2-1.el8 epel 173 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy noarch 18.4.0-1.el8 epel 384 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography x86_64 3.2.1-6.el8 baseos 641 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth noarch 1:1.1.1-10.el8 epel 81 k 2023-10-06T04:28:31.108 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco noarch 6.2-6.el8 epel 11 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools noarch 2.0-4.el8 epel 18 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2 noarch 2.10.1-3.el8 appstream 538 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch noarch 1.21-2.el8 appstream 27 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer noarch 1.10-11.el8 appstream 20 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt noarch 1.6.1-2.el8 baseos 43 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes noarch 1:11.0.0-6.el8 epel 1.2 M 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils noarch 0.3.5-11.el8 epel 49 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako noarch 1.0.6-14.el8 appstream 157 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe x86_64 0.23-19.el8 appstream 39 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools noarch 7.2.0-3.el8 epel 59 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort noarch 7.1.1-2.el8 epel 60 k 2023-10-06T04:28:31.109 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy x86_64 1:1.14.3-10.el8 appstream 3.7 M 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py x86_64 1:1.14.3-10.el8 appstream 225 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib noarch 2.1.0-1.el8 baseos 155 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan noarch 1.3.2-9.el8 epel 283 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend noarch 2.6-1.el8 epel 16 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable noarch 0.7.2-14.el8 appstream 44 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL noarch 19.0.0-1.el8 appstream 103 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1 noarch 0.3.7-6.el8 appstream 126 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules noarch 0.3.7-6.el8 appstream 110 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser noarch 2.14-14.el8 baseos 109 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz noarch 2017.2-11.el8 appstream 57 k 2023-10-06T04:28:31.110 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru noarch 0.7-6.el8 epel 34 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib noarch 1.0.0-1.el8 baseos 43 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes noarch 2.4.1-12.el8 epel 196 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa noarch 4.9-2.el8 epel 64 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy x86_64 1.0.0-21.module_el8.5.0+771+e5d9a225 appstream 14 M 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric noarch 0.8.1-17.el8 epel 19 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch noarch 3.4.0.3-18.el8 epel 24 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora noarch 1.14.1-5.el8 epel 29 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme noarch 0.6.0-4.el8 epel 27 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress noarch 1.4.3-1.el8 epel 255 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob noarch 1.8.5-1.el8.1 epel 251 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client noarch 0.56.0-5.el8 epel 61 k 2023-10-06T04:28:31.111 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest noarch 2.0.33-1.el8 epel 85 k 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug noarch 0.12.2-4.el8 appstream 457 k 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile noarch 2.0-2.el8 epel 23 k 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: socat x86_64 1.7.4.1-1.el8 appstream 323 k 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: thrift x86_64 0.13.0-2.el8 epel 1.7 M 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet x86_64 1.6.1-20.el8 appstream 71 k 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel x86_64 1.2.11-21.el8 baseos 58 k 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction Summary 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout:==================================================================================================== 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout:Install 123 Packages 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout:Upgrade 16 Packages 2023-10-06T04:28:31.112 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:28:31.113 INFO:teuthology.orchestra.run.smithi177.stdout:Total download size: 210 M 2023-10-06T04:28:31.113 INFO:teuthology.orchestra.run.smithi177.stdout:Downloading Packages: 2023-10-06T04:28:31.149 INFO:teuthology.orchestra.run.smithi123.stdout:(10/139): ceph-common-17.2.6-1296.g736ac5ba.el8 7.5 MB/s | 23 MB 00:03 2023-10-06T04:28:31.342 INFO:teuthology.orchestra.run.smithi123.stdout:(11/139): ceph-radosgw-17.2.6-1296.g736ac5ba.el 8.4 MB/s | 17 MB 00:02 2023-10-06T04:28:31.366 INFO:teuthology.orchestra.run.smithi123.stdout:(12/139): cephfs-mirror-17.2.6-1296.g736ac5ba.e 1.2 MB/s | 255 kB 00:00 2023-10-06T04:28:31.392 INFO:teuthology.orchestra.run.smithi123.stdout:(13/139): libcephfs-devel-17.2.6-1296.g736ac5ba 498 kB/s | 25 kB 00:00 2023-10-06T04:28:31.441 INFO:teuthology.orchestra.run.smithi123.stdout:(14/139): libcephsqlite-17.2.6-1296.g736ac5ba.e 3.4 MB/s | 174 kB 00:00 2023-10-06T04:28:31.508 INFO:teuthology.orchestra.run.smithi123.stdout:(15/139): librados-devel-17.2.6-1296.g736ac5ba. 1.9 MB/s | 130 kB 00:00 2023-10-06T04:28:31.551 INFO:teuthology.orchestra.run.smithi123.stdout:(16/139): libcephfs2-17.2.6-1296.g736ac5ba.el8. 4.2 MB/s | 797 kB 00:00 2023-10-06T04:28:31.627 INFO:teuthology.orchestra.run.smithi123.stdout:(17/139): libradosstriper1-17.2.6-1296.g736ac5b 4.2 MB/s | 507 kB 00:00 2023-10-06T04:28:31.711 INFO:teuthology.orchestra.run.smithi123.stdout:(18/139): python3-ceph-argparse-17.2.6-1296.g73 535 kB/s | 45 kB 00:00 2023-10-06T04:28:31.794 INFO:teuthology.orchestra.run.smithi123.stdout:(19/139): python3-ceph-common-17.2.6-1296.g736a 1.4 MB/s | 115 kB 00:00 2023-10-06T04:28:31.911 INFO:teuthology.orchestra.run.smithi123.stdout:(20/139): python3-cephfs-17.2.6-1296.g736ac5ba. 1.6 MB/s | 190 kB 00:00 2023-10-06T04:28:32.224 INFO:teuthology.orchestra.run.smithi177.stdout:(1/139): ceph-17.2.6-1296.g736ac5ba.el8.x86_64. 14 kB/s | 6.4 kB 00:00 2023-10-06T04:28:32.494 INFO:teuthology.orchestra.run.smithi123.stdout:(21/139): librgw2-17.2.6-1296.g736ac5ba.el8.x86 5.9 MB/s | 5.6 MB 00:00 2023-10-06T04:28:32.535 INFO:teuthology.orchestra.run.smithi123.stdout:(22/139): python3-rados-17.2.6-1296.g736ac5ba.e 593 kB/s | 371 kB 00:00 2023-10-06T04:28:32.619 INFO:teuthology.orchestra.run.smithi123.stdout:(23/139): python3-rbd-17.2.6-1296.g736ac5ba.el8 2.8 MB/s | 358 kB 00:00 2023-10-06T04:28:32.645 INFO:teuthology.orchestra.run.smithi123.stdout:(24/139): python3-rgw-17.2.6-1296.g736ac5ba.el8 980 kB/s | 107 kB 00:00 2023-10-06T04:28:32.745 INFO:teuthology.orchestra.run.smithi123.stdout:(25/139): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 722 kB/s | 90 kB 00:00 2023-10-06T04:28:32.824 INFO:teuthology.orchestra.run.smithi177.stdout:(2/139): ceph-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.4 MB/s | 876 kB 00:00 2023-10-06T04:28:32.878 INFO:teuthology.orchestra.run.smithi123.stdout:(26/139): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 1.3 MB/s | 173 kB 00:00 2023-10-06T04:28:32.949 INFO:teuthology.orchestra.run.smithi177.stdout:(3/139): ceph-immutable-object-cache-17.2.6-129 1.3 MB/s | 162 kB 00:00 2023-10-06T04:28:33.020 INFO:teuthology.orchestra.run.smithi123.stdout:(27/139): ceph-grafana-dashboards-17.2.6-1296.g 163 kB/s | 23 kB 00:00 2023-10-06T04:28:33.146 INFO:teuthology.orchestra.run.smithi123.stdout:(28/139): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 970 kB/s | 121 kB 00:00 2023-10-06T04:28:33.354 INFO:teuthology.orchestra.run.smithi123.stdout:(29/139): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 4.9 MB/s | 3.5 MB 00:00 2023-10-06T04:28:33.458 INFO:teuthology.orchestra.run.smithi177.stdout:(4/139): ceph-base-17.2.6-1296.g736ac5ba.el8.x8 3.0 MB/s | 5.2 MB 00:01 2023-10-06T04:28:33.679 INFO:teuthology.orchestra.run.smithi123.stdout:(30/139): ceph-mgr-dashboard-17.2.6-1296.g736ac 3.2 MB/s | 1.7 MB 00:00 2023-10-06T04:28:33.780 INFO:teuthology.orchestra.run.smithi123.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1296.g73 2.3 MB/s | 238 kB 00:00 2023-10-06T04:28:33.863 INFO:teuthology.orchestra.run.smithi123.stdout:(32/139): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 569 kB/s | 47 kB 00:00 2023-10-06T04:28:33.866 INFO:teuthology.orchestra.run.smithi177.stdout:(5/139): ceph-mds-17.2.6-1296.g736ac5ba.el8.x86 2.5 MB/s | 2.3 MB 00:00 2023-10-06T04:28:33.908 INFO:teuthology.orchestra.run.smithi177.stdout:(6/139): ceph-mgr-17.2.6-1296.g736ac5ba.el8.x86 3.5 MB/s | 1.6 MB 00:00 2023-10-06T04:28:33.955 INFO:teuthology.orchestra.run.smithi123.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1296.g7 158 kB/s | 14 kB 00:00 2023-10-06T04:28:34.123 INFO:teuthology.orchestra.run.smithi123.stdout:(34/139): ceph-volume-17.2.6-1296.g736ac5ba.el8 1.6 MB/s | 267 kB 00:00 2023-10-06T04:28:34.384 INFO:teuthology.orchestra.run.smithi123.stdout:(35/139): cephadm-17.2.6-1296.g736ac5ba.el8.noa 309 kB/s | 80 kB 00:00 2023-10-06T04:28:34.499 INFO:teuthology.orchestra.run.smithi123.stdout:(36/139): cephfs-top-17.2.6-1296.g736ac5ba.el8. 158 kB/s | 18 kB 00:00 2023-10-06T04:28:34.713 INFO:teuthology.orchestra.run.smithi123.stdout:(37/139): ceph-mgr-diskprediction-local-17.2.6- 5.4 MB/s | 7.4 MB 00:01 2023-10-06T04:28:34.910 INFO:teuthology.orchestra.run.smithi177.stdout:(7/139): ceph-mon-17.2.6-1296.g736ac5ba.el8.x86 3.8 MB/s | 3.9 MB 00:01 2023-10-06T04:28:35.191 INFO:teuthology.orchestra.run.smithi123.stdout:(38/139): ceph-test-17.2.6-1296.g736ac5ba.el8.x 12 MB/s | 52 MB 00:04 2023-10-06T04:28:35.324 INFO:teuthology.orchestra.run.smithi123.stdout:(39/139): automake-1.16.1-8.el8.noarch.rpm 896 kB/s | 740 kB 00:00 2023-10-06T04:28:35.436 INFO:teuthology.orchestra.run.smithi123.stdout:(40/139): jq-1.6-7.el8.x86_64.rpm 1.8 MB/s | 202 kB 00:00 2023-10-06T04:28:35.516 INFO:teuthology.orchestra.run.smithi123.stdout:(41/139): flex-2.6.1-9.el8.x86_64.rpm 983 kB/s | 320 kB 00:00 2023-10-06T04:28:35.575 INFO:teuthology.orchestra.run.smithi123.stdout:(42/139): librdkafka-0.11.4-3.el8.x86_64.rpm 2.5 MB/s | 354 kB 00:00 2023-10-06T04:28:35.609 INFO:teuthology.orchestra.run.smithi123.stdout:(43/139): bison-3.0.4-10.el8.x86_64.rpm 769 kB/s | 688 kB 00:00 2023-10-06T04:28:35.651 INFO:teuthology.orchestra.run.smithi123.stdout:(44/139): libtool-2.4.6-25.el8.x86_64.rpm 5.2 MB/s | 709 kB 00:00 2023-10-06T04:28:35.684 INFO:teuthology.orchestra.run.smithi123.stdout:(45/139): oniguruma-6.8.2-2.el8.x86_64.rpm 2.4 MB/s | 187 kB 00:00 2023-10-06T04:28:35.734 INFO:teuthology.orchestra.run.smithi123.stdout:(46/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 1.6 MB/s | 259 kB 00:00 2023-10-06T04:28:35.943 INFO:teuthology.orchestra.run.smithi123.stdout:(47/139): openblas-0.3.15-4.el8.x86_64.rpm 16 MB/s | 4.7 MB 00:00 2023-10-06T04:28:35.959 INFO:teuthology.orchestra.run.smithi177.stdout:(8/139): ceph-common-17.2.6-1296.g736ac5ba.el8. 5.6 MB/s | 23 MB 00:04 2023-10-06T04:28:36.068 INFO:teuthology.orchestra.run.smithi123.stdout:(48/139): python3-babel-2.5.1-7.el8.noarch.rpm 14 MB/s | 4.8 MB 00:00 2023-10-06T04:28:36.094 INFO:teuthology.orchestra.run.smithi123.stdout:(49/139): python3-jinja2-2.10.1-3.el8.noarch.rp 3.5 MB/s | 538 kB 00:00 2023-10-06T04:28:36.139 INFO:teuthology.orchestra.run.smithi123.stdout:(50/139): python3-jsonpatch-1.21-2.el8.noarch.r 398 kB/s | 27 kB 00:00 2023-10-06T04:28:36.151 INFO:teuthology.orchestra.run.smithi177.stdout:(9/139): ceph-osd-17.2.6-1296.g736ac5ba.el8.x86 7.5 MB/s | 17 MB 00:02 2023-10-06T04:28:36.161 INFO:teuthology.orchestra.run.smithi123.stdout:(51/139): python3-jsonpointer-1.10-11.el8.noarc 298 kB/s | 20 kB 00:00 2023-10-06T04:28:36.168 INFO:teuthology.orchestra.run.smithi177.stdout:(10/139): ceph-selinux-17.2.6-1296.g736ac5ba.el 117 kB/s | 24 kB 00:00 2023-10-06T04:28:36.203 INFO:teuthology.orchestra.run.smithi123.stdout:(52/139): python3-mako-1.0.6-14.el8.noarch.rpm 2.3 MB/s | 157 kB 00:00 2023-10-06T04:28:36.228 INFO:teuthology.orchestra.run.smithi123.stdout:(53/139): python3-markupsafe-0.23-19.el8.x86_64 580 kB/s | 39 kB 00:00 2023-10-06T04:28:36.234 INFO:teuthology.orchestra.run.smithi177.stdout:(11/139): cephfs-mirror-17.2.6-1296.g736ac5ba.e 3.7 MB/s | 255 kB 00:00 2023-10-06T04:28:36.293 INFO:teuthology.orchestra.run.smithi177.stdout:(12/139): libcephfs-devel-17.2.6-1296.g736ac5ba 425 kB/s | 25 kB 00:00 2023-10-06T04:28:36.320 INFO:teuthology.orchestra.run.smithi123.stdout:(54/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 2.4 MB/s | 225 kB 00:00 2023-10-06T04:28:36.404 INFO:teuthology.orchestra.run.smithi123.stdout:(55/139): python3-prettytable-0.7.2-14.el8.noar 528 kB/s | 44 kB 00:00 2023-10-06T04:28:36.418 INFO:teuthology.orchestra.run.smithi177.stdout:(13/139): libcephfs2-17.2.6-1296.g736ac5ba.el8. 6.2 MB/s | 797 kB 00:00 2023-10-06T04:28:36.463 INFO:teuthology.orchestra.run.smithi123.stdout:(56/139): python3-numpy-1.14.3-10.el8.x86_64.rp 14 MB/s | 3.7 MB 00:00 2023-10-06T04:28:36.468 INFO:teuthology.orchestra.run.smithi177.stdout:(14/139): libcephsqlite-17.2.6-1296.g736ac5ba.e 3.4 MB/s | 174 kB 00:00 2023-10-06T04:28:36.488 INFO:teuthology.orchestra.run.smithi123.stdout:(57/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.2 MB/s | 103 kB 00:00 2023-10-06T04:28:36.530 INFO:teuthology.orchestra.run.smithi123.stdout:(58/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.9 MB/s | 126 kB 00:00 2023-10-06T04:28:36.544 INFO:teuthology.orchestra.run.smithi177.stdout:(15/139): librados-devel-17.2.6-1296.g736ac5ba. 1.7 MB/s | 130 kB 00:00 2023-10-06T04:28:36.613 INFO:teuthology.orchestra.run.smithi123.stdout:(59/139): openblas-threads-0.3.15-4.el8.x86_64. 5.2 MB/s | 4.9 MB 00:00 2023-10-06T04:28:36.636 INFO:teuthology.orchestra.run.smithi177.stdout:(16/139): libradosstriper1-17.2.6-1296.g736ac5b 5.4 MB/s | 507 kB 00:00 2023-10-06T04:28:36.638 INFO:teuthology.orchestra.run.smithi123.stdout:(60/139): python3-pytz-2017.2-11.el8.noarch.rpm 524 kB/s | 57 kB 00:00 2023-10-06T04:28:36.664 INFO:teuthology.orchestra.run.smithi123.stdout:(61/139): python3-pyasn1-modules-0.3.7-6.el8.no 624 kB/s | 110 kB 00:00 2023-10-06T04:28:36.732 INFO:teuthology.orchestra.run.smithi123.stdout:(62/139): python3-werkzeug-0.12.2-4.el8.noarch. 4.9 MB/s | 457 kB 00:00 2023-10-06T04:28:36.757 INFO:teuthology.orchestra.run.smithi123.stdout:(63/139): socat-1.7.4.1-1.el8.x86_64.rpm 3.4 MB/s | 323 kB 00:00 2023-10-06T04:28:36.807 INFO:teuthology.orchestra.run.smithi123.stdout:(64/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 939 kB/s | 71 kB 00:00 2023-10-06T04:28:37.203 INFO:teuthology.orchestra.run.smithi177.stdout:(17/139): librgw2-17.2.6-1296.g736ac5ba.el8.x86 9.8 MB/s | 5.6 MB 00:00 2023-10-06T04:28:37.253 INFO:teuthology.orchestra.run.smithi177.stdout:(18/139): python3-ceph-argparse-17.2.6-1296.g73 898 kB/s | 45 kB 00:00 2023-10-06T04:28:37.303 INFO:teuthology.orchestra.run.smithi177.stdout:(19/139): python3-ceph-common-17.2.6-1296.g736a 2.2 MB/s | 115 kB 00:00 2023-10-06T04:28:37.387 INFO:teuthology.orchestra.run.smithi177.stdout:(20/139): python3-cephfs-17.2.6-1296.g736ac5ba. 2.2 MB/s | 190 kB 00:00 2023-10-06T04:28:37.462 INFO:teuthology.orchestra.run.smithi177.stdout:(21/139): python3-rados-17.2.6-1296.g736ac5ba.e 4.8 MB/s | 371 kB 00:00 2023-10-06T04:28:37.546 INFO:teuthology.orchestra.run.smithi177.stdout:(22/139): python3-rbd-17.2.6-1296.g736ac5ba.el8 4.2 MB/s | 358 kB 00:00 2023-10-06T04:28:37.557 INFO:teuthology.orchestra.run.smithi123.stdout:(65/139): elfutils-libelf-devel-0.189-3.el8.x86 77 kB/s | 62 kB 00:00 2023-10-06T04:28:37.605 INFO:teuthology.orchestra.run.smithi177.stdout:(23/139): python3-rgw-17.2.6-1296.g736ac5ba.el8 1.8 MB/s | 107 kB 00:00 2023-10-06T04:28:37.607 INFO:teuthology.orchestra.run.smithi123.stdout:(66/139): fuse-2.9.7-17.el8.x86_64.rpm 104 kB/s | 83 kB 00:00 2023-10-06T04:28:37.655 INFO:teuthology.orchestra.run.smithi177.stdout:(24/139): rbd-fuse-17.2.6-1296.g736ac5ba.el8.x8 1.8 MB/s | 90 kB 00:00 2023-10-06T04:28:37.824 INFO:teuthology.orchestra.run.smithi123.stdout:(67/139): python3-scipy-1.0.0-21.module_el8.5.0 11 MB/s | 14 MB 00:01 2023-10-06T04:28:38.031 INFO:teuthology.orchestra.run.smithi177.stdout:(25/139): ceph-radosgw-17.2.6-1296.g736ac5ba.el 5.4 MB/s | 17 MB 00:03 2023-10-06T04:28:38.131 INFO:teuthology.orchestra.run.smithi177.stdout:(26/139): rbd-mirror-17.2.6-1296.g736ac5ba.el8. 7.3 MB/s | 3.5 MB 00:00 2023-10-06T04:28:38.133 INFO:teuthology.orchestra.run.smithi123.stdout:(68/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 84 kB/s | 48 kB 00:00 2023-10-06T04:28:38.156 INFO:teuthology.orchestra.run.smithi177.stdout:(27/139): rbd-nbd-17.2.6-1296.g736ac5ba.el8.x86 1.4 MB/s | 173 kB 00:00 2023-10-06T04:28:38.203 INFO:teuthology.orchestra.run.smithi177.stdout:(28/139): ceph-grafana-dashboards-17.2.6-1296.g 326 kB/s | 23 kB 00:00 2023-10-06T04:28:38.240 INFO:teuthology.orchestra.run.smithi177.stdout:(29/139): ceph-mgr-cephadm-17.2.6-1296.g736ac5b 1.4 MB/s | 121 kB 00:00 2023-10-06T04:28:38.266 INFO:teuthology.orchestra.run.smithi123.stdout:(69/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 42 kB/s | 19 kB 00:00 2023-10-06T04:28:38.407 INFO:teuthology.orchestra.run.smithi177.stdout:(30/139): ceph-mgr-dashboard-17.2.6-1296.g736ac 8.3 MB/s | 1.7 MB 00:00 2023-10-06T04:28:38.474 INFO:teuthology.orchestra.run.smithi177.stdout:(31/139): ceph-mgr-modules-core-17.2.6-1296.g73 3.5 MB/s | 238 kB 00:00 2023-10-06T04:28:38.532 INFO:teuthology.orchestra.run.smithi177.stdout:(32/139): ceph-mgr-rook-17.2.6-1296.g736ac5ba.e 813 kB/s | 47 kB 00:00 2023-10-06T04:28:38.591 INFO:teuthology.orchestra.run.smithi177.stdout:(33/139): ceph-prometheus-alerts-17.2.6-1296.g7 248 kB/s | 14 kB 00:00 2023-10-06T04:28:38.666 INFO:teuthology.orchestra.run.smithi177.stdout:(34/139): ceph-volume-17.2.6-1296.g736ac5ba.el8 3.5 MB/s | 267 kB 00:00 2023-10-06T04:28:38.683 INFO:teuthology.orchestra.run.smithi123.stdout:(70/139): libcom_err-devel-1.45.6-5.el8.x86_64. 70 kB/s | 39 kB 00:00 2023-10-06T04:28:38.725 INFO:teuthology.orchestra.run.smithi177.stdout:(35/139): cephadm-17.2.6-1296.g736ac5ba.el8.noa 1.3 MB/s | 80 kB 00:00 2023-10-06T04:28:38.775 INFO:teuthology.orchestra.run.smithi177.stdout:(36/139): cephfs-top-17.2.6-1296.g736ac5ba.el8. 362 kB/s | 18 kB 00:00 2023-10-06T04:28:39.058 INFO:teuthology.orchestra.run.smithi123.stdout:(71/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 386 kB/s | 561 kB 00:01 2023-10-06T04:28:39.067 INFO:teuthology.orchestra.run.smithi177.stdout:(37/139): automake-1.16.1-8.el8.noarch.rpm 2.5 MB/s | 740 kB 00:00 2023-10-06T04:28:39.176 INFO:teuthology.orchestra.run.smithi177.stdout:(38/139): bison-3.0.4-10.el8.x86_64.rpm 6.2 MB/s | 688 kB 00:00 2023-10-06T04:28:39.276 INFO:teuthology.orchestra.run.smithi177.stdout:(39/139): ceph-mgr-diskprediction-local-17.2.6- 7.1 MB/s | 7.4 MB 00:01 2023-10-06T04:28:39.318 INFO:teuthology.orchestra.run.smithi177.stdout:(40/139): flex-2.6.1-9.el8.x86_64.rpm 2.2 MB/s | 320 kB 00:00 2023-10-06T04:28:39.386 INFO:teuthology.orchestra.run.smithi177.stdout:(41/139): librdkafka-0.11.4-3.el8.x86_64.rpm 5.2 MB/s | 354 kB 00:00 2023-10-06T04:28:39.486 INFO:teuthology.orchestra.run.smithi177.stdout:(42/139): libtool-2.4.6-25.el8.x86_64.rpm 6.9 MB/s | 709 kB 00:00 2023-10-06T04:28:39.519 INFO:teuthology.orchestra.run.smithi177.stdout:(43/139): jq-1.6-7.el8.x86_64.rpm 836 kB/s | 202 kB 00:00 2023-10-06T04:28:39.544 INFO:teuthology.orchestra.run.smithi177.stdout:(44/139): lttng-ust-2.8.1-11.el8.x86_64.rpm 4.3 MB/s | 259 kB 00:00 2023-10-06T04:28:39.583 INFO:teuthology.orchestra.run.smithi123.stdout:(72/139): libgfortran-8.5.0-20.el8.x86_64.rpm 490 kB/s | 645 kB 00:01 2023-10-06T04:28:39.620 INFO:teuthology.orchestra.run.smithi177.stdout:(45/139): oniguruma-6.8.2-2.el8.x86_64.rpm 1.8 MB/s | 187 kB 00:00 2023-10-06T04:28:39.728 INFO:teuthology.orchestra.run.smithi177.stdout:(46/139): openblas-0.3.15-4.el8.x86_64.rpm 26 MB/s | 4.7 MB 00:00 2023-10-06T04:28:39.759 INFO:teuthology.orchestra.run.smithi123.stdout:(73/139): libquadmath-8.5.0-20.el8.x86_64.rpm 160 kB/s | 172 kB 00:01 2023-10-06T04:28:39.891 INFO:teuthology.orchestra.run.smithi177.stdout:(47/139): openblas-threads-0.3.15-4.el8.x86_64. 18 MB/s | 4.9 MB 00:00 2023-10-06T04:28:39.962 INFO:teuthology.orchestra.run.smithi177.stdout:(48/139): python3-babel-2.5.1-7.el8.noarch.rpm 21 MB/s | 4.8 MB 00:00 2023-10-06T04:28:39.996 INFO:teuthology.orchestra.run.smithi177.stdout:(49/139): python3-jinja2-2.10.1-3.el8.noarch.rp 5.0 MB/s | 538 kB 00:00 2023-10-06T04:28:40.021 INFO:teuthology.orchestra.run.smithi177.stdout:(50/139): python3-jsonpatch-1.21-2.el8.noarch.r 454 kB/s | 27 kB 00:00 2023-10-06T04:28:40.055 INFO:teuthology.orchestra.run.smithi177.stdout:(51/139): python3-jsonpointer-1.10-11.el8.noarc 340 kB/s | 20 kB 00:00 2023-10-06T04:28:40.080 INFO:teuthology.orchestra.run.smithi177.stdout:(52/139): python3-mako-1.0.6-14.el8.noarch.rpm 2.6 MB/s | 157 kB 00:00 2023-10-06T04:28:40.114 INFO:teuthology.orchestra.run.smithi177.stdout:(53/139): python3-markupsafe-0.23-19.el8.x86_64 661 kB/s | 39 kB 00:00 2023-10-06T04:28:40.173 INFO:teuthology.orchestra.run.smithi177.stdout:(54/139): python3-numpy-f2py-1.14.3-10.el8.x86_ 3.8 MB/s | 225 kB 00:00 2023-10-06T04:28:40.240 INFO:teuthology.orchestra.run.smithi177.stdout:(55/139): python3-numpy-1.14.3-10.el8.x86_64.rp 23 MB/s | 3.7 MB 00:00 2023-10-06T04:28:40.265 INFO:teuthology.orchestra.run.smithi177.stdout:(56/139): python3-prettytable-0.7.2-14.el8.noar 477 kB/s | 44 kB 00:00 2023-10-06T04:28:40.320 INFO:teuthology.orchestra.run.smithi177.stdout:(57/139): python3-pyOpenSSL-19.0.0-1.el8.noarch 1.3 MB/s | 103 kB 00:00 2023-10-06T04:28:40.349 INFO:teuthology.orchestra.run.smithi177.stdout:(58/139): python3-pyasn1-0.3.7-6.el8.noarch.rpm 1.5 MB/s | 126 kB 00:00 2023-10-06T04:28:40.374 INFO:teuthology.orchestra.run.smithi177.stdout:(59/139): python3-pyasn1-modules-0.3.7-6.el8.no 2.0 MB/s | 110 kB 00:00 2023-10-06T04:28:40.408 INFO:teuthology.orchestra.run.smithi177.stdout:(60/139): python3-pytz-2017.2-11.el8.noarch.rpm 971 kB/s | 57 kB 00:00 2023-10-06T04:28:40.417 INFO:teuthology.orchestra.run.smithi123.stdout:(74/139): libsepol-devel-2.9-3.el8.x86_64.rpm 132 kB/s | 87 kB 00:00 2023-10-06T04:28:40.475 INFO:teuthology.orchestra.run.smithi177.stdout:(61/139): python3-werkzeug-0.12.2-4.el8.noarch. 6.7 MB/s | 457 kB 00:00 2023-10-06T04:28:40.542 INFO:teuthology.orchestra.run.smithi177.stdout:(62/139): socat-1.7.4.1-1.el8.x86_64.rpm 4.7 MB/s | 323 kB 00:00 2023-10-06T04:28:40.600 INFO:teuthology.orchestra.run.smithi177.stdout:(63/139): xmlstarlet-1.6.1-20.el8.x86_64.rpm 1.2 MB/s | 71 kB 00:00 2023-10-06T04:28:40.659 INFO:teuthology.orchestra.run.smithi123.stdout:(75/139): libselinux-devel-2.9-8.el8.x86_64.rpm 186 kB/s | 200 kB 00:01 2023-10-06T04:28:40.779 INFO:teuthology.orchestra.run.smithi123.stdout:(76/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 27 kB/s | 47 kB 00:01 2023-10-06T04:28:40.842 INFO:teuthology.orchestra.run.smithi177.stdout:(64/139): python3-scipy-1.0.0-21.module_el8.5.0 29 MB/s | 14 MB 00:00 2023-10-06T04:28:41.076 INFO:teuthology.orchestra.run.smithi123.stdout:(77/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 43 kB/s | 18 kB 00:00 2023-10-06T04:28:41.201 INFO:teuthology.orchestra.run.smithi123.stdout:(78/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 127 kB/s | 100 kB 00:00 2023-10-06T04:28:41.334 INFO:teuthology.orchestra.run.smithi177.stdout:(65/139): ceph-test-17.2.6-1296.g736ac5ba.el8.x 10 MB/s | 52 MB 00:05 2023-10-06T04:28:41.460 INFO:teuthology.orchestra.run.smithi123.stdout:(79/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 64 kB/s | 44 kB 00:00 2023-10-06T04:28:41.518 INFO:teuthology.orchestra.run.smithi177.stdout:(66/139): elfutils-libelf-devel-0.189-3.el8.x86 67 kB/s | 62 kB 00:00 2023-10-06T04:28:41.893 INFO:teuthology.orchestra.run.smithi177.stdout:(67/139): fuse-2.9.7-17.el8.x86_64.rpm 79 kB/s | 83 kB 00:01 2023-10-06T04:28:42.002 INFO:teuthology.orchestra.run.smithi177.stdout:(68/139): keyutils-libs-devel-1.5.10-9.el8.x86_ 72 kB/s | 48 kB 00:00 2023-10-06T04:28:42.294 INFO:teuthology.orchestra.run.smithi177.stdout:(69/139): libaio-devel-0.3.112-1.el8.x86_64.rpm 47 kB/s | 19 kB 00:00 2023-10-06T04:28:42.486 INFO:teuthology.orchestra.run.smithi123.stdout:(80/139): pcre2-devel-10.32-3.el8.x86_64.rpm 471 kB/s | 605 kB 00:01 2023-10-06T04:28:42.570 INFO:teuthology.orchestra.run.smithi177.stdout:(70/139): libcom_err-devel-1.45.6-5.el8.x86_64. 68 kB/s | 39 kB 00:00 2023-10-06T04:28:42.885 INFO:teuthology.orchestra.run.smithi123.stdout:(81/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 1.8 MB/s | 3.2 MB 00:01 2023-10-06T04:28:43.513 INFO:teuthology.orchestra.run.smithi123.stdout:(82/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 112 kB/s | 229 kB 00:02 2023-10-06T04:28:43.811 INFO:teuthology.orchestra.run.smithi123.stdout:(83/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 269 kB/s | 249 kB 00:00 2023-10-06T04:28:44.110 INFO:teuthology.orchestra.run.smithi177.stdout:(71/139): libquadmath-8.5.0-20.el8.x86_64.rpm 112 kB/s | 172 kB 00:01 2023-10-06T04:28:44.361 INFO:teuthology.orchestra.run.smithi123.stdout:(84/139): python3-jwt-1.6.1-2.el8.noarch.rpm 77 kB/s | 43 kB 00:00 2023-10-06T04:28:44.486 INFO:teuthology.orchestra.run.smithi123.stdout:(85/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 110 kB/s | 220 kB 00:02 2023-10-06T04:28:44.827 INFO:teuthology.orchestra.run.smithi177.stdout:(72/139): librabbitmq-0.9.0-4.el8.x86_64.rpm 65 kB/s | 47 kB 00:00 2023-10-06T04:28:44.911 INFO:teuthology.orchestra.run.smithi177.stdout:(73/139): krb5-devel-1.18.2-22.el8.x86_64.rpm 165 kB/s | 561 kB 00:03 2023-10-06T04:28:45.737 INFO:teuthology.orchestra.run.smithi123.stdout:(86/139): python3-pycparser-2.14-14.el8.noarch. 87 kB/s | 109 kB 00:01 2023-10-06T04:28:46.027 INFO:teuthology.orchestra.run.smithi177.stdout:(74/139): libgfortran-8.5.0-20.el8.x86_64.rpm 173 kB/s | 645 kB 00:03 2023-10-06T04:28:46.053 INFO:teuthology.orchestra.run.smithi177.stdout:(75/139): libsepol-devel-2.9-3.el8.x86_64.rpm 76 kB/s | 87 kB 00:01 2023-10-06T04:28:46.078 INFO:teuthology.orchestra.run.smithi123.stdout:(87/139): python3-oauthlib-2.1.0-1.el8.noarch.r 90 kB/s | 155 kB 00:01 2023-10-06T04:28:46.236 INFO:teuthology.orchestra.run.smithi177.stdout:(76/139): libselinux-devel-2.9-8.el8.x86_64.rpm 142 kB/s | 200 kB 00:01 2023-10-06T04:28:46.445 INFO:teuthology.orchestra.run.smithi123.stdout:(88/139): python3-requests-oauthlib-1.0.0-1.el8 60 kB/s | 43 kB 00:00 2023-10-06T04:28:46.486 INFO:teuthology.orchestra.run.smithi177.stdout:(77/139): libverto-devel-0.3.2-2.el8.x86_64.rpm 42 kB/s | 18 kB 00:00 2023-10-06T04:28:46.501 INFO:teuthology.orchestra.run.smithi177.stdout:[MIRROR] libzstd-devel-1.4.4-1.el8.x86_64.rpm: Status code: 503 for http://mirrors.rcs.alaska.edu/centos/8-stream/BaseOS/x86_64/os/Packages/libzstd-devel-1.4.4-1.el8.x86_64.rpm (IP: 199.165.82.202) 2023-10-06T04:28:46.895 INFO:teuthology.orchestra.run.smithi123.stdout:(89/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 91 kB/s | 74 kB 00:00 2023-10-06T04:28:47.162 INFO:teuthology.orchestra.run.smithi177.stdout:(78/139): libuuid-devel-2.32.1-43.el8.x86_64.rp 88 kB/s | 100 kB 00:01 2023-10-06T04:28:47.346 INFO:teuthology.orchestra.run.smithi123.stdout:(90/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 64 kB/s | 58 kB 00:00 2023-10-06T04:28:47.520 INFO:teuthology.orchestra.run.smithi177.stdout:(79/139): libzstd-devel-1.4.4-1.el8.x86_64.rpm 34 kB/s | 44 kB 00:01 2023-10-06T04:28:47.596 INFO:teuthology.orchestra.run.smithi123.stdout:(91/139): python3-cryptography-3.2.1-6.el8.x86_ 157 kB/s | 641 kB 00:04 2023-10-06T04:28:47.730 INFO:teuthology.orchestra.run.smithi123.stdout:(92/139): gperftools-libs-2.7-9.el8.x86_64.rpm 367 kB/s | 306 kB 00:00 2023-10-06T04:28:47.880 INFO:teuthology.orchestra.run.smithi123.stdout:(93/139): liboath-2.6.2-3.el8.x86_64.rpm 110 kB/s | 59 kB 00:00 2023-10-06T04:28:47.989 INFO:teuthology.orchestra.run.smithi123.stdout:(94/139): python3-asyncssh-2.7.0-2.el8.noarch.r 1.8 MB/s | 476 kB 00:00 2023-10-06T04:28:48.039 INFO:teuthology.orchestra.run.smithi123.stdout:(95/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 275 kB/s | 44 kB 00:00 2023-10-06T04:28:48.139 INFO:teuthology.orchestra.run.smithi123.stdout:(96/139): python3-beautifulsoup4-4.6.3-2.el8.1. 1.2 MB/s | 185 kB 00:00 2023-10-06T04:28:48.165 INFO:teuthology.orchestra.run.smithi123.stdout:(97/139): libunwind-1.3.1-3.el8.x86_64.rpm 132 kB/s | 75 kB 00:00 2023-10-06T04:28:48.181 INFO:teuthology.orchestra.run.smithi123.stdout:(98/139): python3-cachetools-3.1.1-4.el8.noarch 232 kB/s | 33 kB 00:00 2023-10-06T04:28:48.257 INFO:teuthology.orchestra.run.smithi123.stdout:(99/139): python3-certifi-2018.10.15-7.el8.noar 139 kB/s | 16 kB 00:00 2023-10-06T04:28:48.399 INFO:teuthology.orchestra.run.smithi123.stdout:(100/139): python3-google-auth-1.1.1-10.el8.noa 573 kB/s | 81 kB 00:00 2023-10-06T04:28:48.441 INFO:teuthology.orchestra.run.smithi123.stdout:(101/139): python3-cheroot-8.5.2-1.el8.noarch.r 626 kB/s | 173 kB 00:00 2023-10-06T04:28:48.516 INFO:teuthology.orchestra.run.smithi123.stdout:(102/139): python3-jaraco-6.2-6.el8.noarch.rpm 95 kB/s | 11 kB 00:00 2023-10-06T04:28:48.566 INFO:teuthology.orchestra.run.smithi123.stdout:(103/139): python3-jaraco-functools-2.0-4.el8.n 140 kB/s | 18 kB 00:00 2023-10-06T04:28:48.591 INFO:teuthology.orchestra.run.smithi123.stdout:(104/139): python3-cherrypy-18.4.0-1.el8.noarch 939 kB/s | 384 kB 00:00 2023-10-06T04:28:48.709 INFO:teuthology.orchestra.run.smithi123.stdout:(105/139): python3-logutils-0.3.5-11.el8.noarch 343 kB/s | 49 kB 00:00 2023-10-06T04:28:48.734 INFO:teuthology.orchestra.run.smithi123.stdout:(106/139): python3-more-itertools-7.2.0-3.el8.n 413 kB/s | 59 kB 00:00 2023-10-06T04:28:48.776 INFO:teuthology.orchestra.run.smithi123.stdout:(107/139): python3-kubernetes-11.0.0-6.el8.noar 4.5 MB/s | 1.2 MB 00:00 2023-10-06T04:28:48.851 INFO:teuthology.orchestra.run.smithi123.stdout:(108/139): python3-natsort-7.1.1-2.el8.noarch.r 424 kB/s | 60 kB 00:00 2023-10-06T04:28:48.894 INFO:teuthology.orchestra.run.smithi123.stdout:(109/139): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-10-06T04:28:48.951 INFO:teuthology.orchestra.run.smithi123.stdout:(110/139): python3-pecan-1.3.2-9.el8.noarch.rpm 1.3 MB/s | 283 kB 00:00 2023-10-06T04:28:48.985 INFO:teuthology.orchestra.run.smithi123.stdout:(111/139): python3-repoze-lru-0.7-6.el8.noarch. 251 kB/s | 34 kB 00:00 2023-10-06T04:28:49.019 INFO:teuthology.orchestra.run.smithi123.stdout:(112/139): python3-routes-2.4.1-12.el8.noarch.r 1.5 MB/s | 196 kB 00:00 2023-10-06T04:28:49.077 INFO:teuthology.orchestra.run.smithi123.stdout:(113/139): python3-rsa-4.9-2.el8.noarch.rpm 512 kB/s | 64 kB 00:00 2023-10-06T04:28:49.079 INFO:teuthology.orchestra.run.smithi177.stdout:(80/139): pcre2-utf16-10.32-3.el8.x86_64.rpm 147 kB/s | 229 kB 00:01 2023-10-06T04:28:49.111 INFO:teuthology.orchestra.run.smithi123.stdout:(114/139): python3-simplegeneric-0.8.1-17.el8.n 149 kB/s | 19 kB 00:00 2023-10-06T04:28:49.136 INFO:teuthology.orchestra.run.smithi123.stdout:(115/139): python3-singledispatch-3.4.0.3-18.el 204 kB/s | 24 kB 00:00 2023-10-06T04:28:49.236 INFO:teuthology.orchestra.run.smithi123.stdout:(116/139): python3-tempora-1.14.1-5.el8.noarch. 184 kB/s | 29 kB 00:00 2023-10-06T04:28:49.262 INFO:teuthology.orchestra.run.smithi123.stdout:(117/139): python3-trustme-0.6.0-4.el8.noarch.r 183 kB/s | 27 kB 00:00 2023-10-06T04:28:49.290 INFO:teuthology.orchestra.run.smithi123.stdout:(118/139): python3-waitress-1.4.3-1.el8.noarch. 1.7 MB/s | 255 kB 00:00 2023-10-06T04:28:49.395 INFO:teuthology.orchestra.run.smithi123.stdout:(119/139): python3-webob-1.8.5-1.el8.1.noarch.r 1.5 MB/s | 251 kB 00:00 2023-10-06T04:28:49.421 INFO:teuthology.orchestra.run.smithi123.stdout:(120/139): python3-websocket-client-0.56.0-5.el 385 kB/s | 61 kB 00:00 2023-10-06T04:28:49.446 INFO:teuthology.orchestra.run.smithi123.stdout:(121/139): python3-webtest-2.0.33-1.el8.noarch. 536 kB/s | 85 kB 00:00 2023-10-06T04:28:49.521 INFO:teuthology.orchestra.run.smithi123.stdout:(122/139): python3-zc-lockfile-2.0-2.el8.noarch 182 kB/s | 23 kB 00:00 2023-10-06T04:28:49.805 INFO:teuthology.orchestra.run.smithi123.stdout:(123/139): librados2-17.2.6-1296.g736ac5ba.el8. 9.7 MB/s | 3.5 MB 00:00 2023-10-06T04:28:49.883 INFO:teuthology.orchestra.run.smithi123.stdout:(124/139): thrift-0.13.0-2.el8.x86_64.rpm 3.8 MB/s | 1.7 MB 00:00 2023-10-06T04:28:50.222 INFO:teuthology.orchestra.run.smithi123.stdout:(125/139): librbd1-17.2.6-1296.g736ac5ba.el8.x8 5.1 MB/s | 3.6 MB 00:00 2023-10-06T04:28:50.555 INFO:teuthology.orchestra.run.smithi123.stdout:[MIRROR] elfutils-libelf-0.189-3.el8.x86_64.rpm: Status code: 503 for http://mirrors.rcs.alaska.edu/centos/8-stream/BaseOS/x86_64/os/Packages/elfutils-libelf-0.189-3.el8.x86_64.rpm (IP: 199.165.82.202) 2023-10-06T04:28:50.621 INFO:teuthology.orchestra.run.smithi177.stdout:(81/139): pcre2-utf32-10.32-3.el8.x86_64.rpm 143 kB/s | 220 kB 00:01 2023-10-06T04:28:50.740 INFO:teuthology.orchestra.run.smithi123.stdout:(126/139): elfutils-debuginfod-client-0.189-3.e 89 kB/s | 76 kB 00:00 2023-10-06T04:28:51.056 INFO:teuthology.orchestra.run.smithi123.stdout:(127/139): NetworkManager-initscripts-updown-1. 115 kB/s | 143 kB 00:01 2023-10-06T04:28:51.121 INFO:teuthology.orchestra.run.smithi177.stdout:(82/139): pcre2-devel-10.32-3.el8.x86_64.rpm 153 kB/s | 605 kB 00:03 2023-10-06T04:28:51.207 INFO:teuthology.orchestra.run.smithi123.stdout:(128/139): elfutils-libelf-0.189-3.el8.x86_64.r 236 kB/s | 232 kB 00:00 2023-10-06T04:28:52.321 INFO:teuthology.orchestra.run.smithi177.stdout:(83/139): python3-cffi-1.11.5-6.el8.x86_64.rpm 146 kB/s | 249 kB 00:01 2023-10-06T04:28:52.588 INFO:teuthology.orchestra.run.smithi123.stdout:(129/139): elfutils-libs-0.189-3.el8.x86_64.rpm 164 kB/s | 303 kB 00:01 2023-10-06T04:28:52.880 INFO:teuthology.orchestra.run.smithi177.stdout:(84/139): python3-jwt-1.6.1-2.el8.noarch.rpm 76 kB/s | 43 kB 00:00 2023-10-06T04:28:52.890 INFO:teuthology.orchestra.run.smithi123.stdout:(130/139): libblkid-2.32.1-43.el8.x86_64.rpm 131 kB/s | 220 kB 00:01 2023-10-06T04:28:53.888 INFO:teuthology.orchestra.run.smithi177.stdout:(85/139): python3-oauthlib-2.1.0-1.el8.noarch.r 154 kB/s | 155 kB 00:01 2023-10-06T04:28:54.140 INFO:teuthology.orchestra.run.smithi123.stdout:(131/139): libfdisk-2.32.1-43.el8.x86_64.rpm 164 kB/s | 254 kB 00:01 2023-10-06T04:28:54.172 INFO:teuthology.orchestra.run.smithi177.stdout:(86/139): python3-cryptography-3.2.1-6.el8.x86_ 210 kB/s | 641 kB 00:03 2023-10-06T04:28:54.307 INFO:teuthology.orchestra.run.smithi123.stdout:(132/139): libmount-2.32.1-43.el8.x86_64.rpm 167 kB/s | 236 kB 00:01 2023-10-06T04:28:54.599 INFO:teuthology.orchestra.run.smithi123.stdout:(133/139): iproute-6.2.0-5.el8.x86_64.rpm 241 kB/s | 854 kB 00:03 2023-10-06T04:28:54.730 INFO:teuthology.orchestra.run.smithi177.stdout:(87/139): python3-requests-oauthlib-1.0.0-1.el8 76 kB/s | 43 kB 00:00 2023-10-06T04:28:54.764 INFO:teuthology.orchestra.run.smithi177.stdout:(88/139): python3-pycparser-2.14-14.el8.noarch. 124 kB/s | 109 kB 00:00 2023-10-06T04:28:55.073 INFO:teuthology.orchestra.run.smithi177.stdout:(89/139): openssl-devel-1.1.1k-9.el8.x86_64.rpm 379 kB/s | 3.2 MB 00:08 2023-10-06T04:28:55.128 INFO:teuthology.orchestra.run.smithi123.stdout:(134/139): libuuid-2.32.1-43.el8.x86_64.rpm 120 kB/s | 99 kB 00:00 2023-10-06T04:28:55.281 INFO:teuthology.orchestra.run.smithi177.stdout:(90/139): zlib-devel-1.2.11-21.el8.x86_64.rpm 112 kB/s | 58 kB 00:00 2023-10-06T04:28:55.358 INFO:teuthology.orchestra.run.smithi123.stdout:(135/139): libsmartcols-2.32.1-43.el8.x86_64.rp 147 kB/s | 179 kB 00:01 2023-10-06T04:28:55.373 INFO:teuthology.orchestra.run.smithi177.stdout:(91/139): xfsprogs-devel-5.0.0-12.el8.x86_64.rp 116 kB/s | 74 kB 00:00 2023-10-06T04:28:55.884 INFO:teuthology.orchestra.run.smithi177.stdout:(92/139): liboath-2.6.2-3.el8.x86_64.rpm 98 kB/s | 59 kB 00:00 2023-10-06T04:28:55.891 INFO:teuthology.orchestra.run.smithi123.stdout:(136/139): util-linux-user-2.32.1-43.el8.x86_64 134 kB/s | 103 kB 00:00 2023-10-06T04:28:55.915 INFO:teuthology.orchestra.run.smithi177.stdout:(93/139): gperftools-libs-2.7-9.el8.x86_64.rpm 363 kB/s | 306 kB 00:00 2023-10-06T04:28:55.940 INFO:teuthology.orchestra.run.smithi177.stdout:(94/139): libunwind-1.3.1-3.el8.x86_64.rpm 133 kB/s | 75 kB 00:00 2023-10-06T04:28:56.033 INFO:teuthology.orchestra.run.smithi177.stdout:(95/139): python3-bcrypt-3.1.6-2.el8.1.x86_64.r 373 kB/s | 44 kB 00:00 2023-10-06T04:28:56.141 INFO:teuthology.orchestra.run.smithi177.stdout:(96/139): python3-cachetools-3.1.1-4.el8.noarch 305 kB/s | 33 kB 00:00 2023-10-06T04:28:56.191 INFO:teuthology.orchestra.run.smithi177.stdout:(97/139): python3-beautifulsoup4-4.6.3-2.el8.1. 739 kB/s | 185 kB 00:00 2023-10-06T04:28:56.233 INFO:teuthology.orchestra.run.smithi177.stdout:(98/139): python3-asyncssh-2.7.0-2.el8.noarch.r 1.3 MB/s | 476 kB 00:00 2023-10-06T04:28:56.250 INFO:teuthology.orchestra.run.smithi177.stdout:(99/139): python3-certifi-2018.10.15-7.el8.noar 150 kB/s | 16 kB 00:00 2023-10-06T04:28:56.350 INFO:teuthology.orchestra.run.smithi177.stdout:(100/139): python3-cheroot-8.5.2-1.el8.noarch.r 1.1 MB/s | 173 kB 00:00 2023-10-06T04:28:56.376 INFO:teuthology.orchestra.run.smithi177.stdout:(101/139): python3-google-auth-1.1.1-10.el8.noa 648 kB/s | 81 kB 00:00 2023-10-06T04:28:56.401 INFO:teuthology.orchestra.run.smithi177.stdout:(102/139): python3-cherrypy-18.4.0-1.el8.noarch 2.2 MB/s | 384 kB 00:00 2023-10-06T04:28:56.459 INFO:teuthology.orchestra.run.smithi177.stdout:(103/139): python3-jaraco-6.2-6.el8.noarch.rpm 102 kB/s | 11 kB 00:00 2023-10-06T04:28:56.485 INFO:teuthology.orchestra.run.smithi177.stdout:(104/139): python3-jaraco-functools-2.0-4.el8.n 161 kB/s | 18 kB 00:00 2023-10-06T04:28:56.585 INFO:teuthology.orchestra.run.smithi177.stdout:(105/139): python3-logutils-0.3.5-11.el8.noarch 389 kB/s | 49 kB 00:00 2023-10-06T04:28:56.610 INFO:teuthology.orchestra.run.smithi177.stdout:(106/139): python3-more-itertools-7.2.0-3.el8.n 469 kB/s | 59 kB 00:00 2023-10-06T04:28:56.652 INFO:teuthology.orchestra.run.smithi177.stdout:(107/139): python3-kubernetes-11.0.0-6.el8.noar 4.6 MB/s | 1.2 MB 00:00 2023-10-06T04:28:56.741 INFO:teuthology.orchestra.run.smithi177.stdout:(108/139): python3-natsort-7.1.1-2.el8.noarch.r 387 kB/s | 60 kB 00:00 2023-10-06T04:28:56.770 INFO:teuthology.orchestra.run.smithi177.stdout:(109/139): python3-portend-2.6-1.el8.noarch.rpm 133 kB/s | 16 kB 00:00 2023-10-06T04:28:56.795 INFO:teuthology.orchestra.run.smithi177.stdout:(110/139): python3-pecan-1.3.2-9.el8.noarch.rpm 1.5 MB/s | 283 kB 00:00 2023-10-06T04:28:56.845 INFO:teuthology.orchestra.run.smithi177.stdout:(111/139): python3-repoze-lru-0.7-6.el8.noarch. 324 kB/s | 34 kB 00:00 2023-10-06T04:28:56.904 INFO:teuthology.orchestra.run.smithi177.stdout:(112/139): python3-rsa-4.9-2.el8.noarch.rpm 591 kB/s | 64 kB 00:00 2023-10-06T04:28:56.929 INFO:teuthology.orchestra.run.smithi177.stdout:(113/139): python3-routes-2.4.1-12.el8.noarch.r 1.2 MB/s | 196 kB 00:00 2023-10-06T04:28:56.954 INFO:teuthology.orchestra.run.smithi177.stdout:(114/139): python3-simplegeneric-0.8.1-17.el8.n 172 kB/s | 19 kB 00:00 2023-10-06T04:28:57.013 INFO:teuthology.orchestra.run.smithi177.stdout:(115/139): python3-singledispatch-3.4.0.3-18.el 220 kB/s | 24 kB 00:00 2023-10-06T04:28:57.038 INFO:teuthology.orchestra.run.smithi177.stdout:(116/139): python3-tempora-1.14.1-5.el8.noarch. 269 kB/s | 29 kB 00:00 2023-10-06T04:28:57.063 INFO:teuthology.orchestra.run.smithi177.stdout:(117/139): python3-trustme-0.6.0-4.el8.noarch.r 253 kB/s | 27 kB 00:00 2023-10-06T04:28:57.175 INFO:teuthology.orchestra.run.smithi123.stdout:(137/139): xfsdump-3.1.8-6.el8.x86_64.rpm 184 kB/s | 334 kB 00:01 2023-10-06T04:28:57.180 INFO:teuthology.orchestra.run.smithi177.stdout:(118/139): python3-waitress-1.4.3-1.el8.noarch. 1.5 MB/s | 255 kB 00:00 2023-10-06T04:28:57.206 INFO:teuthology.orchestra.run.smithi177.stdout:(119/139): python3-websocket-client-0.56.0-5.el 431 kB/s | 61 kB 00:00 2023-10-06T04:28:57.231 INFO:teuthology.orchestra.run.smithi177.stdout:(120/139): python3-webob-1.8.5-1.el8.1.noarch.r 1.3 MB/s | 251 kB 00:00 2023-10-06T04:28:57.299 INFO:teuthology.orchestra.run.smithi177.stdout:(121/139): python3-webtest-2.0.33-1.el8.noarch. 730 kB/s | 85 kB 00:00 2023-10-06T04:28:57.323 INFO:teuthology.orchestra.run.smithi177.stdout:(122/139): python3-zc-lockfile-2.0-2.el8.noarch 196 kB/s | 23 kB 00:00 2023-10-06T04:28:57.599 INFO:teuthology.orchestra.run.smithi177.stdout:(123/139): thrift-0.13.0-2.el8.x86_64.rpm 4.7 MB/s | 1.7 MB 00:00 2023-10-06T04:28:57.715 INFO:teuthology.orchestra.run.smithi177.stdout:(124/139): librbd1-17.2.6-1296.g736ac5ba.el8.x8 9.1 MB/s | 3.6 MB 00:00 2023-10-06T04:28:57.908 INFO:teuthology.orchestra.run.smithi177.stdout:(125/139): librados2-17.2.6-1296.g736ac5ba.el8. 5.7 MB/s | 3.5 MB 00:00 2023-10-06T04:28:58.424 INFO:teuthology.orchestra.run.smithi177.stdout:(126/139): elfutils-debuginfod-client-0.189-3.e 107 kB/s | 76 kB 00:00 2023-10-06T04:28:58.541 INFO:teuthology.orchestra.run.smithi177.stdout:(127/139): NetworkManager-initscripts-updown-1. 152 kB/s | 143 kB 00:00 2023-10-06T04:28:58.850 INFO:teuthology.orchestra.run.smithi177.stdout:(128/139): elfutils-libelf-0.189-3.el8.x86_64.r 246 kB/s | 232 kB 00:00 2023-10-06T04:28:59.441 INFO:teuthology.orchestra.run.smithi123.stdout:(138/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 312 kB/s | 1.1 MB 00:03 2023-10-06T04:28:59.625 INFO:teuthology.orchestra.run.smithi177.stdout:(129/139): elfutils-libs-0.189-3.el8.x86_64.rpm 252 kB/s | 303 kB 00:01 2023-10-06T04:28:59.892 INFO:teuthology.orchestra.run.smithi177.stdout:(130/139): libblkid-2.32.1-43.el8.x86_64.rpm 212 kB/s | 220 kB 00:01 2023-10-06T04:28:59.975 INFO:teuthology.orchestra.run.smithi177.stdout:(131/139): iproute-6.2.0-5.el8.x86_64.rpm 595 kB/s | 854 kB 00:01 2023-10-06T04:28:59.992 INFO:teuthology.orchestra.run.smithi123.stdout:(139/139): util-linux-2.32.1-43.el8.x86_64.rpm 470 kB/s | 2.5 MB 00:05 2023-10-06T04:28:59.995 INFO:teuthology.orchestra.run.smithi123.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:28:59.995 INFO:teuthology.orchestra.run.smithi123.stdout:Total 6.5 MB/s | 210 MB 00:32 2023-10-06T04:29:00.634 INFO:teuthology.orchestra.run.smithi177.stdout:(132/139): libfdisk-2.32.1-43.el8.x86_64.rpm 252 kB/s | 254 kB 00:01 2023-10-06T04:29:00.826 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction check 2023-10-06T04:29:00.867 INFO:teuthology.orchestra.run.smithi177.stdout:(133/139): libsmartcols-2.32.1-43.el8.x86_64.rp 201 kB/s | 179 kB 00:00 2023-10-06T04:29:00.934 INFO:teuthology.orchestra.run.smithi177.stdout:(134/139): libmount-2.32.1-43.el8.x86_64.rpm 227 kB/s | 236 kB 00:01 2023-10-06T04:29:01.250 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction check succeeded. 2023-10-06T04:29:01.251 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction test 2023-10-06T04:29:01.493 INFO:teuthology.orchestra.run.smithi177.stdout:(135/139): libuuid-2.32.1-43.el8.x86_64.rpm 115 kB/s | 99 kB 00:00 2023-10-06T04:29:01.710 INFO:teuthology.orchestra.run.smithi177.stdout:(136/139): util-linux-user-2.32.1-43.el8.x86_64 132 kB/s | 103 kB 00:00 2023-10-06T04:29:02.652 INFO:teuthology.orchestra.run.smithi177.stdout:(137/139): xfsdump-3.1.8-6.el8.x86_64.rpm 288 kB/s | 334 kB 00:01 2023-10-06T04:29:02.680 INFO:teuthology.orchestra.run.smithi123.stdout:Transaction test succeeded. 2023-10-06T04:29:02.683 INFO:teuthology.orchestra.run.smithi123.stdout:Running transaction 2023-10-06T04:29:03.327 INFO:teuthology.orchestra.run.smithi177.stdout:(138/139): xfsprogs-5.0.0-12.el8.x86_64.rpm 685 kB/s | 1.1 MB 00:01 2023-10-06T04:29:04.178 INFO:teuthology.orchestra.run.smithi177.stdout:(139/139): util-linux-2.32.1-43.el8.x86_64.rpm 766 kB/s | 2.5 MB 00:03 2023-10-06T04:29:04.180 INFO:teuthology.orchestra.run.smithi177.stdout:-------------------------------------------------------------------------------- 2023-10-06T04:29:04.181 INFO:teuthology.orchestra.run.smithi177.stdout:Total 6.3 MB/s | 210 MB 00:33 2023-10-06T04:29:05.016 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction check 2023-10-06T04:29:05.441 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction check succeeded. 2023-10-06T04:29:05.441 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction test 2023-10-06T04:29:06.840 INFO:teuthology.orchestra.run.smithi177.stdout:Transaction test succeeded. 2023-10-06T04:29:06.843 INFO:teuthology.orchestra.run.smithi177.stdout:Running transaction 2023-10-06T04:29:08.277 INFO:teuthology.orchestra.run.smithi123.stdout: Preparing : 1/1 2023-10-06T04:29:08.322 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-06T04:29:08.473 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T04:29:08.583 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T04:29:08.748 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T04:29:08.857 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T04:29:09.055 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-06T04:29:09.435 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-06T04:29:09.627 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-06T04:29:09.852 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-06T04:29:10.072 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T04:29:10.436 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T04:29:10.639 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/155 2023-10-06T04:29:11.037 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/155 2023-10-06T04:29:11.267 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/155 2023-10-06T04:29:11.377 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/155 2023-10-06T04:29:11.597 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/155 2023-10-06T04:29:11.690 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/155 2023-10-06T04:29:11.904 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/155 2023-10-06T04:29:12.105 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T04:29:12.161 INFO:teuthology.orchestra.run.smithi177.stdout: Preparing : 1/1 2023-10-06T04:29:12.200 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/1 2023-10-06T04:29:12.390 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T04:29:12.468 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libuuid-2.32.1-43.el8.x86_64 1/155 2023-10-06T04:29:12.656 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T04:29:12.666 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T04:29:12.726 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libblkid-2.32.1-43.el8.x86_64 2/155 2023-10-06T04:29:12.789 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/155 2023-10-06T04:29:12.938 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : liboath-2.6.2-3.el8.x86_64 3/155 2023-10-06T04:29:13.034 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/155 2023-10-06T04:29:13.289 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T04:29:13.309 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libelf-0.189-3.el8.x86_64 4/155 2023-10-06T04:29:13.363 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T04:29:13.537 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : thrift-0.13.0-2.el8.x86_64 5/155 2023-10-06T04:29:13.586 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-06T04:29:13.720 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librabbitmq-0.9.0-4.el8.x86_64 6/155 2023-10-06T04:29:13.779 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-06T04:29:13.922 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 17/155 2023-10-06T04:29:13.923 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T04:29:14.033 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 17/155 2023-10-06T04:29:14.163 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/155 2023-10-06T04:29:14.286 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/155 2023-10-06T04:29:14.305 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: lttng-ust-2.8.1-11.el8.x86_64 7/155 2023-10-06T04:29:14.499 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/155 2023-10-06T04:29:14.511 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-06T04:29:14.693 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-06T04:29:14.830 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T04:29:14.894 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/155 2023-10-06T04:29:14.981 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T04:29:15.121 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T04:29:15.128 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/155 2023-10-06T04:29:15.254 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/155 2023-10-06T04:29:15.432 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/155 2023-10-06T04:29:15.525 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/155 2023-10-06T04:29:15.736 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T04:29:15.746 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/155 2023-10-06T04:29:15.890 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T04:29:15.963 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T04:29:16.473 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librdkafka-0.11.4-3.el8.x86_64 12/155 2023-10-06T04:29:16.605 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T04:29:16.615 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/155 2023-10-06T04:29:16.763 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T04:29:17.014 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/155 2023-10-06T04:29:17.348 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T04:29:17.406 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: xfsprogs-5.0.0-12.el8.x86_64 14/155 2023-10-06T04:29:17.454 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T04:29:17.612 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-more-itertools-7.2.0-3.el8.noarch 15/155 2023-10-06T04:29:17.799 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-06T04:29:17.830 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-0.3.7-6.el8.noarch 16/155 2023-10-06T04:29:17.965 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 17/155 2023-10-06T04:29:18.019 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-06T04:29:18.067 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 17/155 2023-10-06T04:29:18.184 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-06T04:29:18.206 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/155 2023-10-06T04:29:18.321 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/155 2023-10-06T04:29:18.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-06T04:29:18.545 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webob-1.8.5-1.el8.1.noarch 19/155 2023-10-06T04:29:18.644 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 29/155 2023-10-06T04:29:18.709 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : zlib-devel-1.2.11-21.el8.x86_64 20/155 2023-10-06T04:29:18.807 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 30/155 2023-10-06T04:29:18.839 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T04:29:18.954 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libquadmath-8.5.0-20.el8.x86_64 21/155 2023-10-06T04:29:19.040 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 31/155 2023-10-06T04:29:19.106 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T04:29:19.707 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libgfortran-8.5.0-20.el8.x86_64 22/155 2023-10-06T04:29:19.965 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T04:29:20.012 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-06T04:29:20.371 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-06T04:29:20.574 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-0.3.15-4.el8.x86_64 23/155 2023-10-06T04:29:20.660 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-06T04:29:20.747 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T04:29:20.862 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T04:29:21.443 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: openblas-threads-0.3.15-4.el8.x86_64 24/155 2023-10-06T04:29:21.750 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-1:1.14.3-10.el8.x86_64 25/155 2023-10-06T04:29:21.962 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-werkzeug-0.12.2-4.el8.noarch 26/155 2023-10-06T04:29:22.101 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pytz-2017.2-11.el8.noarch 27/155 2023-10-06T04:29:22.325 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-markupsafe-0.23-19.el8.x86_64 28/155 2023-10-06T04:29:22.537 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 29/155 2023-10-06T04:29:22.650 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T04:29:22.745 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 30/155 2023-10-06T04:29:22.953 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 31/155 2023-10-06T04:29:22.983 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-06T04:29:23.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-06T04:29:23.441 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-06T04:29:23.639 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/155 2023-10-06T04:29:23.825 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 40/155 2023-10-06T04:29:23.929 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-mako-1.0.6-14.el8.noarch 32/155 2023-10-06T04:29:24.070 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 41/155 2023-10-06T04:29:24.269 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-06T04:29:24.271 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-babel-2.5.1-7.el8.noarch 33/155 2023-10-06T04:29:24.437 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-06T04:29:24.569 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jinja2-2.10.1-3.el8.noarch 34/155 2023-10-06T04:29:24.670 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T04:29:24.747 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T04:29:24.788 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T04:29:24.995 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T04:29:25.047 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T04:29:25.225 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-06T04:29:25.428 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T04:29:25.969 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T04:29:26.270 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T04:29:26.285 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T04:29:26.481 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-06T04:29:26.594 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 35/155 2023-10-06T04:29:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-06T04:29:26.917 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 36/155 2023-10-06T04:29:27.001 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-06T04:29:27.114 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyasn1-modules-0.3.7-6.el8.noarch 37/155 2023-10-06T04:29:27.207 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-06T04:29:27.341 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rsa-4.9-2.el8.noarch 38/155 2023-10-06T04:29:27.413 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-06T04:29:27.523 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 39/155 2023-10-06T04:29:27.594 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-06T04:29:27.726 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 40/155 2023-10-06T04:29:27.831 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-06T04:29:27.949 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 41/155 2023-10-06T04:29:28.038 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-06T04:29:28.165 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-libs-0.189-3.el8.x86_64 42/155 2023-10-06T04:29:28.214 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-06T04:29:28.330 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : elfutils-debuginfod-client-0.189-3.el8.x86_64 43/155 2023-10-06T04:29:28.375 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-06T04:29:28.554 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T04:29:28.565 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-06T04:29:28.632 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libfdisk-2.32.1-43.el8.x86_64 44/155 2023-10-06T04:29:28.750 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-06T04:29:28.813 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T04:29:28.883 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libmount-2.32.1-43.el8.x86_64 45/155 2023-10-06T04:29:28.953 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-06T04:29:29.059 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libuuid-devel-2.32.1-43.el8.x86_64 46/155 2023-10-06T04:29:29.146 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-06T04:29:29.229 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T04:29:29.354 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-06T04:29:29.628 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-06T04:29:29.748 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libsmartcols-2.32.1-43.el8.x86_64 47/155 2023-10-06T04:29:29.882 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-06T04:29:30.005 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T04:29:30.020 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: util-linux-2.32.1-43.el8.x86_64 48/155 2023-10-06T04:29:30.182 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-zc-lockfile-2.0-2.el8.noarch 49/155 2023-10-06T04:29:30.242 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-06T04:29:30.439 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-websocket-client-0.56.0-5.el8.noarch 50/155 2023-10-06T04:29:30.495 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-06T04:29:30.652 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-waitress-1.4.3-1.el8.noarch 51/155 2023-10-06T04:29:30.711 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-06T04:29:30.792 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-singledispatch-3.4.0.3-18.el8.noarch 52/155 2023-10-06T04:29:30.931 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-simplegeneric-0.8.1-17.el8.noarch 53/155 2023-10-06T04:29:30.948 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-06T04:29:31.145 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-repoze-lru-0.7-6.el8.noarch 54/155 2023-10-06T04:29:31.197 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-06T04:29:31.340 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-routes-2.4.1-12.el8.noarch 55/155 2023-10-06T04:29:31.523 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-natsort-7.1.1-2.el8.noarch 56/155 2023-10-06T04:29:31.530 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-06T04:29:31.657 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-logutils-0.3.5-11.el8.noarch 57/155 2023-10-06T04:29:31.809 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-6.2-6.el8.noarch 58/155 2023-10-06T04:29:31.822 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-06T04:29:32.007 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jaraco-functools-2.0-4.el8.noarch 59/155 2023-10-06T04:29:32.062 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-06T04:29:32.159 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-tempora-1.14.1-5.el8.noarch 60/155 2023-10-06T04:29:32.281 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-06T04:29:32.354 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-portend-2.6-1.el8.noarch 61/155 2023-10-06T04:29:32.506 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-06T04:29:32.533 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-certifi-2018.10.15-7.el8.noarch 62/155 2023-10-06T04:29:32.750 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cachetools-3.1.1-4.el8.noarch 63/155 2023-10-06T04:29:32.979 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-google-auth-1:1.1.1-10.el8.noarch 64/155 2023-10-06T04:29:33.200 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 65/155 2023-10-06T04:29:33.528 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-webtest-2.0.33-1.el8.noarch 66/155 2023-10-06T04:29:33.611 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-06T04:29:33.746 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pecan-1.3.2-9.el8.noarch 67/155 2023-10-06T04:29:33.928 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libunwind-1.3.1-3.el8.x86_64 68/155 2023-10-06T04:29:33.980 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-06T04:29:34.166 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : gperftools-libs-1:2.7-9.el8.x86_64 69/155 2023-10-06T04:29:34.185 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-06T04:29:34.431 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pycparser-2.14-14.el8.noarch 70/155 2023-10-06T04:29:34.455 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-06T04:29:34.751 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-06T04:29:34.790 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cffi-1.11.5-6.el8.x86_64 71/155 2023-10-06T04:29:34.970 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-06T04:29:35.057 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cryptography-3.2.1-6.el8.x86_64 72/155 2023-10-06T04:29:35.246 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-06T04:29:35.280 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-pyOpenSSL-19.0.0-1.el8.noarch 73/155 2023-10-06T04:29:35.497 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T04:29:35.518 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jwt-1.6.1-2.el8.noarch 74/155 2023-10-06T04:29:35.767 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-oauthlib-2.1.0-1.el8.noarch 75/155 2023-10-06T04:29:35.804 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-06T04:29:36.057 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-06T04:29:36.212 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-06T04:29:36.424 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-06T04:29:36.805 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-requests-oauthlib-1.0.0-1.el8.noarch 76/155 2023-10-06T04:29:37.156 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-kubernetes-1:11.0.0-6.el8.noarch 77/155 2023-10-06T04:29:37.156 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-06T04:29:37.403 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-asyncssh-2.7.0-2.el8.noarch 78/155 2023-10-06T04:29:37.414 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-06T04:29:37.582 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-06T04:29:37.652 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-trustme-0.6.0-4.el8.noarch 79/155 2023-10-06T04:29:37.838 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-06T04:29:38.010 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cheroot-8.5.2-1.el8.noarch 80/155 2023-10-06T04:29:38.070 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-06T04:29:38.254 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-cherrypy-18.4.0-1.el8.noarch 81/155 2023-10-06T04:29:38.270 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-06T04:29:38.497 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-bcrypt-3.1.6-2.el8.1.x86_64 82/155 2023-10-06T04:29:38.520 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-06T04:29:38.730 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T04:29:38.735 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-06T04:29:38.877 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-06T04:29:39.027 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-utf16-10.32-3.el8.x86_64 84/155 2023-10-06T04:29:39.258 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : pcre2-devel-10.32-3.el8.x86_64 85/155 2023-10-06T04:29:39.438 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libzstd-devel-1.4.4-1.el8.x86_64 86/155 2023-10-06T04:29:39.641 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libverto-devel-0.3.2-2.el8.x86_64 87/155 2023-10-06T04:29:39.998 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libsepol-devel-2.9-3.el8.x86_64 88/155 2023-10-06T04:29:40.214 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libselinux-devel-2.9-8.el8.x86_64 89/155 2023-10-06T04:29:40.356 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcom_err-devel-1.45.6-5.el8.x86_64 90/155 2023-10-06T04:29:40.556 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : keyutils-libs-devel-1.5.10-9.el8.x86_64 91/155 2023-10-06T04:29:40.787 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : krb5-devel-1.18.2-22.el8.x86_64 92/155 2023-10-06T04:29:40.978 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : fuse-2.9.7-17.el8.x86_64 93/155 2023-10-06T04:29:41.229 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xmlstarlet-1.6.1-20.el8.x86_64 94/155 2023-10-06T04:29:41.410 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : socat-1.7.4.1-1.el8.x86_64 95/155 2023-10-06T04:29:41.590 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 97/155 2023-10-06T04:29:41.594 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-prettytable-0.7.2-14.el8.noarch 96/155 2023-10-06T04:29:41.843 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 97/155 2023-10-06T04:29:42.430 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 97/155 2023-10-06T04:29:43.061 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 98/155 2023-10-06T04:29:43.173 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 98/155 2023-10-06T04:29:43.400 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 99/155 2023-10-06T04:29:44.233 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 97/155 2023-10-06T04:29:44.510 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 97/155 2023-10-06T04:29:45.084 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 97/155 2023-10-06T04:29:45.398 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 98/155 2023-10-06T04:29:45.499 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 98/155 2023-10-06T04:29:45.668 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 99/155 2023-10-06T04:30:01.896 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 99/155 2023-10-06T04:30:01.896 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /sys 2023-10-06T04:30:01.896 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /proc 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /run 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /mnt 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /var/tmp 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /home 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /tmp 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout:skipping the directory /dev 2023-10-06T04:30:01.897 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:02.299 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 100/155 2023-10-06T04:30:03.770 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 100/155 2023-10-06T04:30:04.024 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/155 2023-10-06T04:30:04.201 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 99/155 2023-10-06T04:30:04.201 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /sys 2023-10-06T04:30:04.201 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /proc 2023-10-06T04:30:04.201 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-10-06T04:30:04.201 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /run 2023-10-06T04:30:04.202 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /mnt 2023-10-06T04:30:04.202 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /var/tmp 2023-10-06T04:30:04.202 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /home 2023-10-06T04:30:04.202 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /tmp 2023-10-06T04:30:04.202 INFO:teuthology.orchestra.run.smithi177.stdout:skipping the directory /dev 2023-10-06T04:30:04.202 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:04.245 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/155 2023-10-06T04:30:04.422 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 102/155 2023-10-06T04:30:04.590 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 100/155 2023-10-06T04:30:04.820 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 102/155 2023-10-06T04:30:05.006 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/155 2023-10-06T04:30:05.029 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/155 2023-10-06T04:30:05.229 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-06T04:30:05.466 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-06T04:30:05.630 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T04:30:05.706 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T04:30:05.982 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-06T04:30:06.062 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 100/155 2023-10-06T04:30:06.205 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-06T04:30:06.353 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/155 2023-10-06T04:30:06.574 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 101/155 2023-10-06T04:30:06.740 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 109/155 2023-10-06T04:30:06.756 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 102/155 2023-10-06T04:30:06.942 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 109/155 2023-10-06T04:30:07.111 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 110/155 2023-10-06T04:30:07.171 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 102/155 2023-10-06T04:30:07.291 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 111/155 2023-10-06T04:30:07.356 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/155 2023-10-06T04:30:07.380 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 103/155 2023-10-06T04:30:07.513 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 112/155 2023-10-06T04:30:07.605 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpointer-1.10-11.el8.noarch 104/155 2023-10-06T04:30:07.724 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 112/155 2023-10-06T04:30:07.809 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : python3-jsonpatch-1.21-2.el8.noarch 105/155 2023-10-06T04:30:07.939 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 113/155 2023-10-06T04:30:07.973 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T04:30:08.051 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: oniguruma-6.8.2-2.el8.x86_64 106/155 2023-10-06T04:30:08.325 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : jq-1.6-7.el8.x86_64 107/155 2023-10-06T04:30:08.539 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : automake-1.16.1-8.el8.noarch 108/155 2023-10-06T04:30:08.902 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 113/155 2023-10-06T04:30:09.092 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 109/155 2023-10-06T04:30:09.113 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 114/155 2023-10-06T04:30:09.203 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 114/155 2023-10-06T04:30:09.272 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 109/155 2023-10-06T04:30:09.412 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 110/155 2023-10-06T04:30:09.586 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 115/155 2023-10-06T04:30:09.627 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 111/155 2023-10-06T04:30:09.806 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/155 2023-10-06T04:30:09.841 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/155 2023-10-06T04:30:09.847 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 112/155 2023-10-06T04:30:10.054 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 117/155 2023-10-06T04:30:10.063 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 112/155 2023-10-06T04:30:10.065 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 117/155 2023-10-06T04:30:10.248 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 118/155 2023-10-06T04:30:10.268 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 113/155 2023-10-06T04:30:10.404 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T04:30:11.233 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 113/155 2023-10-06T04:30:11.421 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 114/155 2023-10-06T04:30:11.514 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 114/155 2023-10-06T04:30:11.898 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 115/155 2023-10-06T04:30:12.127 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/155 2023-10-06T04:30:12.163 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 116/155 2023-10-06T04:30:12.363 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 117/155 2023-10-06T04:30:12.374 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 117/155 2023-10-06T04:30:12.612 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 118/155 2023-10-06T04:30:12.763 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T04:30:15.321 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T04:30:17.031 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 120/155 2023-10-06T04:30:17.256 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/155 2023-10-06T04:30:17.373 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/155 2023-10-06T04:30:17.537 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 122/155 2023-10-06T04:30:17.578 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 122/155 2023-10-06T04:30:17.629 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libtool-2.4.6-25.el8.x86_64 119/155 2023-10-06T04:30:17.754 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 123/155 2023-10-06T04:30:18.099 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 123/155 2023-10-06T04:30:18.287 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 124/155 2023-10-06T04:30:18.381 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 124/155 2023-10-06T04:30:19.250 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 125/155 2023-10-06T04:30:19.385 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 120/155 2023-10-06T04:30:19.557 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-06T04:30:19.659 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/155 2023-10-06T04:30:19.759 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-06T04:30:19.763 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 121/155 2023-10-06T04:30:19.970 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 122/155 2023-10-06T04:30:19.972 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-06T04:30:20.013 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: ceph-immutable-object-cache-2:17.2.6-1296.g736ac 122/155 2023-10-06T04:30:20.173 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-06T04:30:20.187 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 123/155 2023-10-06T04:30:20.463 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 130/155 2023-10-06T04:30:20.534 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 123/155 2023-10-06T04:30:20.708 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-06T04:30:20.721 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 124/155 2023-10-06T04:30:20.815 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 124/155 2023-10-06T04:30:20.886 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 132/155 2023-10-06T04:30:21.166 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 133/155 2023-10-06T04:30:21.485 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 134/155 2023-10-06T04:30:21.682 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 125/155 2023-10-06T04:30:21.763 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-06T04:30:21.974 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : openssl-devel-1:1.1.1k-9.el8.x86_64 126/155 2023-10-06T04:30:21.977 INFO:teuthology.orchestra.run.smithi123.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T04:30:21.993 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T04:30:22.184 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : elfutils-libelf-devel-0.189-3.el8.x86_64 127/155 2023-10-06T04:30:22.247 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-06T04:30:22.356 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : util-linux-user-2.32.1-43.el8.x86_64 128/155 2023-10-06T04:30:22.494 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T04:30:22.573 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : xfsprogs-devel-5.0.0-12.el8.x86_64 129/155 2023-10-06T04:30:22.582 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T04:30:22.795 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 130/155 2023-10-06T04:30:22.828 INFO:teuthology.orchestra.run.smithi123.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T04:30:22.841 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T04:30:23.054 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-06T04:30:23.074 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : xfsdump-3.1.8-6.el8.x86_64 131/155 2023-10-06T04:30:23.214 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-06T04:30:23.320 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 132/155 2023-10-06T04:30:23.414 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-06T04:30:23.414 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T04:30:23.583 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 133/155 2023-10-06T04:30:23.688 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T04:30:23.689 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T04:30:23.934 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 134/155 2023-10-06T04:30:23.938 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T04:30:23.943 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T04:30:24.072 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T04:30:24.073 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T04:30:24.222 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : iproute-6.2.0-5.el8.x86_64 135/155 2023-10-06T04:30:24.308 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T04:30:24.345 INFO:teuthology.orchestra.run.smithi177.stdout: Upgrading : NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T04:30:24.359 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-9.el 136/155 2023-10-06T04:30:24.512 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-06T04:30:24.582 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : libaio-devel-0.3.112-1.el8.x86_64 137/155 2023-10-06T04:30:24.657 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-06T04:30:24.803 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T04:30:24.871 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T04:30:24.871 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T04:30:24.898 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: flex-2.6.1-9.el8.x86_64 138/155 2023-10-06T04:30:25.096 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T04:30:25.097 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T04:30:25.153 INFO:teuthology.orchestra.run.smithi177.stdout: Installing : bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T04:30:25.166 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: bison-3.0.4-10.el8.x86_64 139/155 2023-10-06T04:30:25.304 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T04:30:25.305 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T04:30:25.363 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-debuginfod-client-0.189-1.el8.x86_64 140/155 2023-10-06T04:30:25.521 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T04:30:25.521 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T04:30:25.542 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libs-0.189-1.el8.x86_64 141/155 2023-10-06T04:30:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsdump-3.1.8-4.el8.x86_64 142/155 2023-10-06T04:30:25.708 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T04:30:25.771 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T04:30:25.771 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T04:30:25.979 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T04:30:25.997 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: xfsprogs-5.0.0-10.el8.x86_64 143/155 2023-10-06T04:30:25.997 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T04:30:26.170 INFO:teuthology.orchestra.run.smithi123.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T04:30:26.204 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librbd1-1:12.2.7-9.el8.x86_64 144/155 2023-10-06T04:30:26.210 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T04:30:26.356 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : NetworkManager-initscripts-updown-1:1.40.16-4.el 145/155 2023-10-06T04:30:26.356 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T04:30:26.575 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: librados2-1:12.2.7-9.el8.x86_64 146/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/155 2023-10-06T04:30:26.756 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/155 2023-10-06T04:30:26.757 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/155 2023-10-06T04:30:26.758 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-06T04:30:26.759 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-06T04:30:26.760 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-06T04:30:26.761 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-06T04:30:26.762 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-06T04:30:26.763 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-06T04:30:26.764 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-06T04:30:26.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-06T04:30:26.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-06T04:30:26.767 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-06T04:30:26.768 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-06T04:30:26.769 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 124/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 126/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-06T04:30:26.770 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-06T04:30:26.771 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-06T04:30:26.772 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-06T04:30:26.773 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-06T04:30:26.773 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-06T04:30:26.773 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-06T04:30:26.773 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : iproute-6.2.0-1.el8.x86_64 147/155 2023-10-06T04:30:26.924 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-user-2.32.1-41.el8.x86_64 148/155 2023-10-06T04:30:27.171 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T04:30:27.171 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T04:30:27.380 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libfdisk-2.32.1-41.el8.x86_64 150/155 2023-10-06T04:30:27.380 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T04:30:27.588 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libmount-2.32.1-41.el8.x86_64 151/155 2023-10-06T04:30:27.588 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T04:30:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libblkid-2.32.1-41.el8.x86_64 152/155 2023-10-06T04:30:27.788 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T04:30:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libuuid-2.32.1-41.el8.x86_64 153/155 2023-10-06T04:30:28.013 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T04:30:28.205 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: libsmartcols-2.32.1-41.el8.x86_64 154/155 2023-10-06T04:30:28.386 INFO:teuthology.orchestra.run.smithi177.stdout: Cleanup : elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T04:30:28.977 INFO:teuthology.orchestra.run.smithi177.stdout: Running scriptlet: elfutils-libelf-0.189-1.el8.x86_64 155/155 2023-10-06T04:30:28.977 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 1/155 2023-10-06T04:30:28.977 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2/155 2023-10-06T04:30:28.977 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 3/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 4/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-immutable-object-cache-2:17.2.6-1296.g736ac 5/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 6/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 7/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 8/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 9/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 10/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 11/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 12/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 13/155 2023-10-06T04:30:28.978 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_ 14/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 15/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 16/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_6 17/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86 18/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 19/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el 20/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8. 21/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_6 22/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 23/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 24/155 2023-10-06T04:30:28.979 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 25/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 26/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 27/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 28/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba. 29/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noa 30/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.n 31/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-diskprediction-local-2:17.2.6-1296.g736 32/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el 33/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 34/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.e 35/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 36/155 2023-10-06T04:30:28.980 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 37/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 38/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : automake-1.16.1-8.el8.noarch 39/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : bison-3.0.4-10.el8.x86_64 40/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : flex-2.6.1-9.el8.x86_64 41/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : jq-1.6-7.el8.x86_64 42/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librdkafka-0.11.4-3.el8.x86_64 43/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libtool-2.4.6-25.el8.x86_64 44/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : lttng-ust-2.8.1-11.el8.x86_64 45/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : oniguruma-6.8.2-2.el8.x86_64 46/155 2023-10-06T04:30:28.981 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-0.3.15-4.el8.x86_64 47/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openblas-threads-0.3.15-4.el8.x86_64 48/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-babel-2.5.1-7.el8.noarch 49/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jinja2-2.10.1-3.el8.noarch 50/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpatch-1.21-2.el8.noarch 51/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jsonpointer-1.10-11.el8.noarch 52/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-mako-1.0.6-14.el8.noarch 53/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-markupsafe-0.23-19.el8.x86_64 54/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-1:1.14.3-10.el8.x86_64 55/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-numpy-f2py-1:1.14.3-10.el8.x86_64 56/155 2023-10-06T04:30:28.982 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-prettytable-0.7.2-14.el8.noarch 57/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyOpenSSL-19.0.0-1.el8.noarch 58/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-0.3.7-6.el8.noarch 59/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pyasn1-modules-0.3.7-6.el8.noarch 60/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pytz-2017.2-11.el8.noarch 61/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a2 62/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-werkzeug-0.12.2-4.el8.noarch 63/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : socat-1.7.4.1-1.el8.x86_64 64/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xmlstarlet-1.6.1-20.el8.x86_64 65/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-devel-0.189-3.el8.x86_64 66/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : fuse-2.9.7-17.el8.x86_64 67/155 2023-10-06T04:30:28.983 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : keyutils-libs-devel-1.5.10-9.el8.x86_64 68/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : krb5-devel-1.18.2-22.el8.x86_64 69/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libaio-devel-0.3.112-1.el8.x86_64 70/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libcom_err-devel-1.45.6-5.el8.x86_64 71/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libgfortran-8.5.0-20.el8.x86_64 72/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libquadmath-8.5.0-20.el8.x86_64 73/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librabbitmq-0.9.0-4.el8.x86_64 74/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libselinux-devel-2.9-8.el8.x86_64 75/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsepol-devel-2.9-3.el8.x86_64 76/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-devel-2.32.1-43.el8.x86_64 77/155 2023-10-06T04:30:28.984 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libverto-devel-0.3.2-2.el8.x86_64 78/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libzstd-devel-1.4.4-1.el8.x86_64 79/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : openssl-devel-1:1.1.1k-9.el8.x86_64 80/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-devel-10.32-3.el8.x86_64 81/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf16-10.32-3.el8.x86_64 82/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : pcre2-utf32-10.32-3.el8.x86_64 83/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cffi-1.11.5-6.el8.x86_64 84/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cryptography-3.2.1-6.el8.x86_64 85/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jwt-1.6.1-2.el8.noarch 86/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-oauthlib-2.1.0-1.el8.noarch 87/155 2023-10-06T04:30:28.985 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pycparser-2.14-14.el8.noarch 88/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-requests-oauthlib-1.0.0-1.el8.noarch 89/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-devel-5.0.0-12.el8.x86_64 90/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : zlib-devel-1.2.11-21.el8.x86_64 91/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : gperftools-libs-1:2.7-9.el8.x86_64 92/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : liboath-2.6.2-3.el8.x86_64 93/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libunwind-1.3.1-3.el8.x86_64 94/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-asyncssh-2.7.0-2.el8.noarch 95/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-bcrypt-3.1.6-2.el8.1.x86_64 96/155 2023-10-06T04:30:28.986 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-beautifulsoup4-4.6.3-2.el8.1.noarch 97/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cachetools-3.1.1-4.el8.noarch 98/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-certifi-2018.10.15-7.el8.noarch 99/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cheroot-8.5.2-1.el8.noarch 100/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-cherrypy-18.4.0-1.el8.noarch 101/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-google-auth-1:1.1.1-10.el8.noarch 102/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-6.2-6.el8.noarch 103/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-jaraco-functools-2.0-4.el8.noarch 104/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-kubernetes-1:11.0.0-6.el8.noarch 105/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-logutils-0.3.5-11.el8.noarch 106/155 2023-10-06T04:30:28.987 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-more-itertools-7.2.0-3.el8.noarch 107/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-natsort-7.1.1-2.el8.noarch 108/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-pecan-1.3.2-9.el8.noarch 109/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-portend-2.6-1.el8.noarch 110/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-repoze-lru-0.7-6.el8.noarch 111/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-routes-2.4.1-12.el8.noarch 112/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-rsa-4.9-2.el8.noarch 113/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-simplegeneric-0.8.1-17.el8.noarch 114/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-singledispatch-3.4.0.3-18.el8.noarch 115/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-tempora-1.14.1-5.el8.noarch 116/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-trustme-0.6.0-4.el8.noarch 117/155 2023-10-06T04:30:28.988 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-waitress-1.4.3-1.el8.noarch 118/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webob-1.8.5-1.el8.1.noarch 119/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-websocket-client-0.56.0-5.el8.noarch 120/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-webtest-2.0.33-1.el8.noarch 121/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : python3-zc-lockfile-2.0-2.el8.noarch 122/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : thrift-0.13.0-2.el8.x86_64 123/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 124/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librados2-1:12.2.7-9.el8.x86_64 125/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 126/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : librbd1-1:12.2.7-9.el8.x86_64 127/155 2023-10-06T04:30:28.989 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-9.el 128/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : NetworkManager-initscripts-updown-1:1.40.16-4.el 129/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.189-3.el8.x86_64 130/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-debuginfod-client-0.189-1.el8.x86_64 131/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.189-3.el8.x86_64 132/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libelf-0.189-1.el8.x86_64 133/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.189-3.el8.x86_64 134/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : elfutils-libs-0.189-1.el8.x86_64 135/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-6.2.0-5.el8.x86_64 136/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : iproute-6.2.0-1.el8.x86_64 137/155 2023-10-06T04:30:28.990 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.32.1-43.el8.x86_64 138/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libblkid-2.32.1-41.el8.x86_64 139/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.32.1-43.el8.x86_64 140/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libfdisk-2.32.1-41.el8.x86_64 141/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.32.1-43.el8.x86_64 142/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libmount-2.32.1-41.el8.x86_64 143/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.32.1-43.el8.x86_64 144/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libsmartcols-2.32.1-41.el8.x86_64 145/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.32.1-43.el8.x86_64 146/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : libuuid-2.32.1-41.el8.x86_64 147/155 2023-10-06T04:30:28.991 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.32.1-43.el8.x86_64 148/155 2023-10-06T04:30:28.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-2.32.1-41.el8.x86_64 149/155 2023-10-06T04:30:28.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.32.1-43.el8.x86_64 150/155 2023-10-06T04:30:28.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : util-linux-user-2.32.1-41.el8.x86_64 151/155 2023-10-06T04:30:28.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.8-6.el8.x86_64 152/155 2023-10-06T04:30:28.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsdump-3.1.8-4.el8.x86_64 153/155 2023-10-06T04:30:28.992 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-5.0.0-12.el8.x86_64 154/155 2023-10-06T04:30:29.006 INFO:teuthology.orchestra.run.smithi123.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-06T04:30:29.006 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout:Upgraded: 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.007 INFO:teuthology.orchestra.run.smithi123.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout:Installed: 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: automake-1.16.1-8.el8.noarch 2023-10-06T04:30:29.008 INFO:teuthology.orchestra.run.smithi123.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.009 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.010 INFO:teuthology.orchestra.run.smithi123.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: jq-1.6-7.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-06T04:30:29.011 INFO:teuthology.orchestra.run.smithi123.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.012 INFO:teuthology.orchestra.run.smithi123.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-06T04:30:29.013 INFO:teuthology.orchestra.run.smithi123.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T04:30:29.014 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T04:30:29.015 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T04:30:29.016 INFO:teuthology.orchestra.run.smithi123.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.017 INFO:teuthology.orchestra.run.smithi123.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T04:30:29.018 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-06T04:30:29.019 INFO:teuthology.orchestra.run.smithi123.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T04:30:29.020 INFO:teuthology.orchestra.run.smithi123.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-06T04:30:29.020 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:29.020 INFO:teuthology.orchestra.run.smithi123.stdout:Complete! 2023-10-06T04:30:29.255 DEBUG:teuthology.parallel:result is None 2023-10-06T04:30:31.179 INFO:teuthology.orchestra.run.smithi177.stdout: Verifying : xfsprogs-5.0.0-10.el8.x86_64 155/155 2023-10-06T04:30:31.179 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:31.179 INFO:teuthology.orchestra.run.smithi177.stdout:Upgraded: 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: NetworkManager-initscripts-updown-1:1.40.16-9.el8.noarch 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-debuginfod-client-0.189-3.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-0.189-3.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libs-0.189-3.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: iproute-6.2.0-5.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: libblkid-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: libfdisk-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: libmount-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: librados2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: librbd1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: libsmartcols-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.180 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: util-linux-user-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: xfsdump-3.1.8-6.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-5.0.0-12.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout:Installed: 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: automake-1.16.1-8.el8.noarch 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: bison-3.0.4-10.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-base-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.181 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-grafana-dashboards-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-immutable-object-cache-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mds-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-dashboard-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-diskprediction-local-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-modules-core-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mgr-rook-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-mon-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-osd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.182 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-prometheus-alerts-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-radosgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-selinux-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-test-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: ceph-volume-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: cephadm-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: cephfs-top-2:17.2.6-1296.g736ac5ba.el8.noarch 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: elfutils-libelf-devel-0.189-3.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: flex-2.6.1-9.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: fuse-2.9.7-17.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: gperftools-libs-1:2.7-9.el8.x86_64 2023-10-06T04:30:31.183 INFO:teuthology.orchestra.run.smithi177.stdout: jq-1.6-7.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: keyutils-libs-devel-1.5.10-9.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: krb5-devel-1.18.2-22.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: libaio-devel-0.3.112-1.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: libcephfs2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: libcephsqlite-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: libcom_err-devel-1.45.6-5.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: libgfortran-8.5.0-20.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: liboath-2.6.2-3.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: libquadmath-8.5.0-20.el8.x86_64 2023-10-06T04:30:31.184 INFO:teuthology.orchestra.run.smithi177.stdout: librabbitmq-0.9.0-4.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: librados-devel-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libradosstriper1-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: librdkafka-0.11.4-3.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: librgw2-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libselinux-devel-2.9-8.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libsepol-devel-2.9-3.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libtool-2.4.6-25.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libunwind-1.3.1-3.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libuuid-devel-2.32.1-43.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libverto-devel-0.3.2-2.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: libzstd-devel-1.4.4-1.el8.x86_64 2023-10-06T04:30:31.185 INFO:teuthology.orchestra.run.smithi177.stdout: lttng-ust-2.8.1-11.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: oniguruma-6.8.2-2.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-0.3.15-4.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: openblas-threads-0.3.15-4.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: openssl-devel-1:1.1.1k-9.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-devel-10.32-3.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf16-10.32-3.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: pcre2-utf32-10.32-3.el8.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: python3-asyncssh-2.7.0-2.el8.noarch 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: python3-babel-2.5.1-7.el8.noarch 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: python3-bcrypt-3.1.6-2.el8.1.x86_64 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: python3-beautifulsoup4-4.6.3-2.el8.1.noarch 2023-10-06T04:30:31.186 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cachetools-3.1.1-4.el8.noarch 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-argparse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-ceph-common-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cephfs-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-certifi-2018.10.15-7.el8.noarch 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cffi-1.11.5-6.el8.x86_64 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cheroot-8.5.2-1.el8.noarch 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cherrypy-18.4.0-1.el8.noarch 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-cryptography-3.2.1-6.el8.x86_64 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-google-auth-1:1.1.1-10.el8.noarch 2023-10-06T04:30:31.187 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-6.2-6.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jaraco-functools-2.0-4.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jinja2-2.10.1-3.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpatch-1.21-2.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jsonpointer-1.10-11.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-jwt-1.6.1-2.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-kubernetes-1:11.0.0-6.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-logutils-0.3.5-11.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-mako-1.0.6-14.el8.noarch 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-markupsafe-0.23-19.el8.x86_64 2023-10-06T04:30:31.188 INFO:teuthology.orchestra.run.smithi177.stdout: python3-more-itertools-7.2.0-3.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-natsort-7.1.1-2.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-1:1.14.3-10.el8.x86_64 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-numpy-f2py-1:1.14.3-10.el8.x86_64 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-oauthlib-2.1.0-1.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pecan-1.3.2-9.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-portend-2.6-1.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-prettytable-0.7.2-14.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyOpenSSL-19.0.0-1.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-0.3.7-6.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pyasn1-modules-0.3.7-6.el8.noarch 2023-10-06T04:30:31.189 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pycparser-2.14-14.el8.noarch 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-pytz-2017.2-11.el8.noarch 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rados-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-repoze-lru-0.7-6.el8.noarch 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-requests-oauthlib-1.0.0-1.el8.noarch 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rgw-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-routes-2.4.1-12.el8.noarch 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-rsa-4.9-2.el8.noarch 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-scipy-1.0.0-21.module_el8.5.0+771+e5d9a225.x86_64 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-simplegeneric-0.8.1-17.el8.noarch 2023-10-06T04:30:31.190 INFO:teuthology.orchestra.run.smithi177.stdout: python3-singledispatch-3.4.0.3-18.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-tempora-1.14.1-5.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-trustme-0.6.0-4.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-waitress-1.4.3-1.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webob-1.8.5-1.el8.1.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-websocket-client-0.56.0-5.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-webtest-2.0.33-1.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-werkzeug-0.12.2-4.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: python3-zc-lockfile-2.0-2.el8.noarch 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-fuse-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-mirror-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: rbd-nbd-2:17.2.6-1296.g736ac5ba.el8.x86_64 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: socat-1.7.4.1-1.el8.x86_64 2023-10-06T04:30:31.191 INFO:teuthology.orchestra.run.smithi177.stdout: thrift-0.13.0-2.el8.x86_64 2023-10-06T04:30:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: xfsprogs-devel-5.0.0-12.el8.x86_64 2023-10-06T04:30:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: xmlstarlet-1.6.1-20.el8.x86_64 2023-10-06T04:30:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: zlib-devel-1.2.11-21.el8.x86_64 2023-10-06T04:30:31.192 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:31.192 INFO:teuthology.orchestra.run.smithi177.stdout:Complete! 2023-10-06T04:30:31.461 DEBUG:teuthology.parallel:result is None 2023-10-06T04:30:31.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T04:30:31.688 DEBUG:teuthology.orchestra.run.smithi123:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T04:30:31.716 INFO:teuthology.orchestra.run.smithi123.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T04:30:31.717 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T04:30:31.717 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T04:30:31.717 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F8%2Fx86_64&sha1=736ac5ba72cb47af7d6f6666b1e100f7e20a20e2 2023-10-06T04:30:31.959 DEBUG:teuthology.orchestra.run.smithi177:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2023-10-06T04:30:31.983 INFO:teuthology.orchestra.run.smithi177.stdout:17.2.6-1296.g736ac5ba.el8 2023-10-06T04:30:31.983 INFO:teuthology.packaging:The installed version of ceph is 17.2.6-1296.g736ac5ba.el8 2023-10-06T04:30:31.983 INFO:teuthology.task.install:The correct ceph version 17.2.6-1296.g736ac5ba is installed. 2023-10-06T04:30:31.984 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-10-06T04:30:31.984 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:31.984 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T04:30:32.021 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:32.021 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-10-06T04:30:32.055 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-10-06T04:30:32.055 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:32.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T04:30:32.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T04:30:32.162 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:32.162 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/daemon-helper 2023-10-06T04:30:32.190 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-10-06T04:30:32.257 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-10-06T04:30:32.258 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:32.258 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T04:30:32.287 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T04:30:32.357 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:32.357 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/adjust-ulimits 2023-10-06T04:30:32.384 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-10-06T04:30:32.452 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-10-06T04:30:32.452 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:32.453 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T04:30:32.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T04:30:32.548 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:32.548 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/usr/bin/stdin-killer 2023-10-06T04:30:32.577 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-10-06T04:30:32.644 INFO:teuthology.run_tasks:Running task ceph... 2023-10-06T04:30:32.728 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-10-06T04:30:32.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 777 /var/log/ceph 2023-10-06T04:30:32.730 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 777 /var/log/ceph 2023-10-06T04:30:32.757 INFO:tasks.ceph:Disabling ceph logrotate... 2023-10-06T04:30:32.757 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T04:30:32.799 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/logrotate.d/ceph 2023-10-06T04:30:32.826 INFO:tasks.ceph:Creating extra log directories... 2023-10-06T04:30:32.826 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T04:30:32.867 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-10-06T04:30:32.901 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-10-06T04:30:32.901 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-10-06T04:30:32.902 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-10-06T04:30:32.902 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:32.902 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T04:30:32.950 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:33.064 DEBUG:teuthology.orchestra.run.smithi123:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:33.090 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:33.090 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-10-06T04:30:33.108 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:33.221 DEBUG:teuthology.orchestra.run.smithi177:> sudo chcon system_u:object_r:etc_t:s0 /etc/logrotate.d/ceph-test.conf 2023-10-06T04:30:33.253 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-10-06T04:30:33.253 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-10-06T04:30:33.254 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270', 'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool}', 'email': 'yweinste@redhat.com', 'first_in_suite': False, 'job_id': '7412270', 'kernel': {'kdb': True, 'sha1': 'distro'}, 'ktype': 'distro', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'centos', 'os_version': '8.stream', 'overrides': {'admin_socket': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'bad backtrace', 'object missing on disk', 'error reading table object', 'error reading sessionmap', 'unmatched fragstat', 'unmatched rstat', 'was unreadable, recreating it now', 'Scrub error on inode', 'Metadata damage detected', 'MDS_FAILED', 'MDS_DAMAGE'], 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, 'selinux': {'whitelist': ['scontext=system_u:system_r:logrotate_t:s0']}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f'}}, 'owner': 'scheduled_yuriw@teuthology', 'priority': 75, 'repo': 'https://github.com/ceph/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 7519, 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2', 'sleep_before_teardown': 0, 'subset': '111/12000', 'suite': 'fs', 'suite_branch': 'wip-yuri-testing-2023-10-05-0730-quincy', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph-c_f1e315a9732d36c05ec316678e2397dc58561c7f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph-ci.git', 'suite_sha1': 'f1e315a9732d36c05ec316678e2397dc58561c7f', 'targets': {'smithi123.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNp26BaAn7SY4GNBkQ7fC3roi2Fy5jDHELVp1aQAnmwKkLpQ6giX1JCjpJCFE1YDQ/lLsVn0wDCnUbMkRM0TRoc=', 'smithi177.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB2u8XPPwgdHORuEefQ8WR+QZ56B5fuFI/t8L6agrYh8Ue1xj0j0+zirQ3XAVbygcIz91+KnMCvpSIxGAfzXA54='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel']}, 'flavor': 'default', 'sha1': '736ac5ba72cb47af7d6f6666b1e100f7e20a20e2'}}, {'ceph': None}, {'ceph-fuse': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_recovery_pool']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ['local function is_kupstream() return false end', 'local function is_kdistro() return false end', 'local function is_fuse() return true end', "local function syntax_version() return '' end"]}, 'teuthology_branch': 'main', 'teuthology_sha1': '54e62bcbac4e53d9685e08328b790d3b20d71cae', 'timestamp': '2023-10-05_23:52:50', 'tube': 'smithi', 'user': 'yuriw', 'verbose': True, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.2830'} 2023-10-06T04:30:33.257 DEBUG:teuthology.orchestra.run.smithi123:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T04:30:33.259 DEBUG:teuthology.orchestra.run.smithi177:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-10-06T04:30:33.306 DEBUG:teuthology.orchestra.run.smithi123:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T04:30:33.317 DEBUG:teuthology.orchestra.run.smithi177:> sudo install -d -m0777 -- /var/run/ceph 2023-10-06T04:30:33.373 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:33.374 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T04:30:33.400 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T04:30:33.401 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_1 2023-10-06T04:30:33.458 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 397548 Links: 1 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-06 04:30:26.381753933 +0000 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-06 04:27:12.939548812 +0000 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-06 04:27:12.939548812 +0000 2023-10-06T04:30:33.459 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-06T04:30:33.460 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T04:30:33.525 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-06T04:30:33.525 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-06T04:30:33.525 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000222953 s, 2.3 MB/s 2023-10-06T04:30:33.526 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T04:30:33.585 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_2 2023-10-06T04:30:33.643 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T04:30:33.643 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:33.643 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 390826 Links: 1 2023-10-06T04:30:33.644 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:33.644 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:33.644 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-06 04:30:26.381753933 +0000 2023-10-06T04:30:33.644 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-06 04:27:13.470536645 +0000 2023-10-06T04:30:33.644 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-06 04:27:13.470536645 +0000 2023-10-06T04:30:33.644 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-06T04:30:33.645 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T04:30:33.709 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-06T04:30:33.709 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-06T04:30:33.709 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000216565 s, 2.4 MB/s 2023-10-06T04:30:33.710 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T04:30:33.769 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_3 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 396742 Links: 1 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-06 04:30:26.381753933 +0000 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-06 04:27:13.975525074 +0000 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-06 04:27:13.975525074 +0000 2023-10-06T04:30:33.827 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-06T04:30:33.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T04:30:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-06T04:30:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-06T04:30:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.000215542 s, 2.4 MB/s 2023-10-06T04:30:33.894 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T04:30:33.953 DEBUG:teuthology.orchestra.run.smithi123:> stat /dev/vg_nvme/lv_4 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout:Device: 6h/6d Inode: 397881 Links: 1 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout:Access: 2023-10-06 04:30:26.381753933 +0000 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout:Modify: 2023-10-06 04:27:14.497513113 +0000 2023-10-06T04:30:34.011 INFO:teuthology.orchestra.run.smithi123.stdout:Change: 2023-10-06 04:27:14.497513113 +0000 2023-10-06T04:30:34.012 INFO:teuthology.orchestra.run.smithi123.stdout: Birth: - 2023-10-06T04:30:34.012 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T04:30:34.077 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records in 2023-10-06T04:30:34.077 INFO:teuthology.orchestra.run.smithi123.stderr:1+0 records out 2023-10-06T04:30:34.077 INFO:teuthology.orchestra.run.smithi123.stderr:512 bytes copied, 0.00025093 s, 2.0 MB/s 2023-10-06T04:30:34.078 DEBUG:teuthology.orchestra.run.smithi123:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T04:30:34.136 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:34.137 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:34.137 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/scratch_devs of=/dev/stdout 2023-10-06T04:30:34.156 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-10-06T04:30:34.156 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_1 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 392988 Links: 1 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 04:30:28.604596559 +0000 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 04:27:12.929512533 +0000 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 04:27:12.929512533 +0000 2023-10-06T04:30:34.216 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T04:30:34.217 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-10-06T04:30:34.283 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T04:30:34.283 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T04:30:34.283 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000238146 s, 2.1 MB/s 2023-10-06T04:30:34.284 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-10-06T04:30:34.344 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_2 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 396638 Links: 1 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 04:30:28.604596559 +0000 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 04:27:13.462498369 +0000 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 04:27:13.462498369 +0000 2023-10-06T04:30:34.402 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T04:30:34.403 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-10-06T04:30:34.468 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T04:30:34.468 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T04:30:34.468 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000233043 s, 2.2 MB/s 2023-10-06T04:30:34.469 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-10-06T04:30:34.527 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_3 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 393175 Links: 1 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 04:30:28.604596559 +0000 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 04:27:13.982484550 +0000 2023-10-06T04:30:34.586 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 04:27:13.982484550 +0000 2023-10-06T04:30:34.587 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T04:30:34.587 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-10-06T04:30:34.651 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T04:30:34.652 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T04:30:34.652 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000212898 s, 2.4 MB/s 2023-10-06T04:30:34.653 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-10-06T04:30:34.711 DEBUG:teuthology.orchestra.run.smithi177:> stat /dev/vg_nvme/lv_4 2023-10-06T04:30:34.769 INFO:teuthology.orchestra.run.smithi177.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout:Device: 6h/6d Inode: 387069 Links: 1 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout:Context: system_u:object_r:device_t:s0 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout:Access: 2023-10-06 04:30:28.604596559 +0000 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout:Modify: 2023-10-06 04:27:14.492470997 +0000 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout:Change: 2023-10-06 04:27:14.492470997 +0000 2023-10-06T04:30:34.770 INFO:teuthology.orchestra.run.smithi177.stdout: Birth: - 2023-10-06T04:30:34.771 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-10-06T04:30:34.835 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records in 2023-10-06T04:30:34.836 INFO:teuthology.orchestra.run.smithi177.stderr:1+0 records out 2023-10-06T04:30:34.836 INFO:teuthology.orchestra.run.smithi177.stderr:512 bytes copied, 0.000233651 s, 2.2 MB/s 2023-10-06T04:30:34.837 DEBUG:teuthology.orchestra.run.smithi177:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-10-06T04:30:34.896 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:34.896 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T04:30:34.896 INFO:tasks.ceph:Generating config... 2023-10-06T04:30:34.897 INFO:tasks.ceph:[client] client mount timeout = 600 2023-10-06T04:30:34.897 INFO:tasks.ceph:[client] debug client = 20 2023-10-06T04:30:34.897 INFO:tasks.ceph:[client] debug ms = 1 2023-10-06T04:30:34.897 INFO:tasks.ceph:[client] pid file = 2023-10-06T04:30:34.897 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-10-06T04:30:34.898 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] debug mds = 20 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] debug ms = 1 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] mds debug frag = True 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-10-06T04:30:34.898 INFO:tasks.ceph:[mgr] debug ms = 1 2023-10-06T04:30:34.899 INFO:tasks.ceph:[mon] debug mon = 20 2023-10-06T04:30:34.899 INFO:tasks.ceph:[mon] debug ms = 1 2023-10-06T04:30:34.899 INFO:tasks.ceph:[mon] debug paxos = 20 2023-10-06T04:30:34.899 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] bdev async discard = True 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] debug ms = 1 2023-10-06T04:30:34.899 INFO:tasks.ceph:[osd] debug osd = 20 2023-10-06T04:30:34.900 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-10-06T04:30:34.900 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-10-06T04:30:34.900 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-10-06T04:30:34.900 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-10-06T04:30:34.900 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-10-06T04:30:34.900 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-10-06T04:30:34.900 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-10-06T04:30:34.900 INFO:tasks.ceph:Setting up mon.a... 2023-10-06T04:30:34.900 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-10-06T04:30:34.959 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.keyring 2023-10-06T04:30:34.962 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-10-06T04:30:35.005 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T04:30:35.033 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.123'), ('mon.b', '172.21.15.177'), ('mon.c', '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]')] 2023-10-06T04:30:35.033 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.123,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-10-06T04:30:35.034 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:35.035 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T04:30:35.092 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.123 --add b 172.21.15.177 --addv c '[v2:172.21.15.177:3301,v1:172.21.15.177:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: generated fsid 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:30:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:setting min_mon_release = octopus 2023-10-06T04:30:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:epoch 0 2023-10-06T04:30:35.178 INFO:teuthology.orchestra.run.smithi123.stdout:fsid 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:30:35.179 INFO:teuthology.orchestra.run.smithi123.stdout:last_changed 2023-10-06T04:30:35.177090+0000 2023-10-06T04:30:35.179 INFO:teuthology.orchestra.run.smithi123.stdout:created 2023-10-06T04:30:35.177090+0000 2023-10-06T04:30:35.179 INFO:teuthology.orchestra.run.smithi123.stdout:min_mon_release 15 (octopus) 2023-10-06T04:30:35.179 INFO:teuthology.orchestra.run.smithi123.stdout:election_strategy: 1 2023-10-06T04:30:35.179 INFO:teuthology.orchestra.run.smithi123.stdout:0: [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] mon.a 2023-10-06T04:30:35.179 INFO:teuthology.orchestra.run.smithi123.stdout:1: [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] mon.b 2023-10-06T04:30:35.180 INFO:teuthology.orchestra.run.smithi123.stdout:2: [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] mon.c 2023-10-06T04:30:35.180 INFO:teuthology.orchestra.run.smithi123.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-10-06T04:30:35.182 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-10-06T04:30:35.199 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1cea82d3-d2ef-4e20-8c3e-67296f7df92c... 2023-10-06T04:30:35.201 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:30:35.241 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.284 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2023-10-06T04:30:35.285 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2023-10-06T04:30:35.286 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2023-10-06T04:30:35.287 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.123,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2023-10-06T04:30:35.288 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2023-10-06T04:30:35.289 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2023-10-06T04:30:35.290 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: bdev async discard = True 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: bdev enable discard = True 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore allocator = bitmap 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 1/20 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 1/20 2023-10-06T04:30:35.291 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 4/10 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.292 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2023-10-06T04:30:35.293 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2023-10-06T04:30:35.294 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:30:35.295 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2023-10-06T04:30:35.296 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-10-06T04:30:35.297 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = True 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-10-06T04:30:35.298 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-10-06T04:30:35.299 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2023-10-06T04:30:35.302 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:30:35.303 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2023-10-06T04:30:35.304 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2023-10-06T04:30:35.305 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.123,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2023-10-06T04:30:35.306 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2023-10-06T04:30:35.307 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2023-10-06T04:30:35.308 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:30:35.309 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:30:35.310 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2023-10-06T04:30:35.311 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:30:35.312 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2023-10-06T04:30:35.313 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2023-10-06T04:30:35.314 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = True 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T04:30:35.315 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T04:30:35.315 INFO:tasks.ceph:Creating admin key on mon.a... 2023-10-06T04:30:35.316 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-10-06T04:30:35.395 INFO:tasks.ceph:Copying monmap to all nodes... 2023-10-06T04:30:35.396 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:35.396 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-10-06T04:30:35.413 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:35.413 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-10-06T04:30:35.471 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:30:35.471 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:35.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T04:30:35.471 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T04:30:35.549 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:30:35.549 DEBUG:teuthology.orchestra.run.smithi123:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:35.607 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:30:35.607 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:35.607 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd of=/etc/ceph/ceph.keyring 2023-10-06T04:30:35.607 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-10-06T04:30:35.646 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:30:35.646 DEBUG:teuthology.orchestra.run.smithi177:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:35.703 INFO:tasks.ceph:Setting up mon nodes... 2023-10-06T04:30:35.704 INFO:tasks.ceph:Setting up mgr nodes... 2023-10-06T04:30:35.704 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T04:30:35.754 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-10-06T04:30:35.757 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T04:30:35.819 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-10-06T04:30:35.822 INFO:tasks.ceph:Setting up mds nodes... 2023-10-06T04:30:35.822 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T04:30:35.872 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-10-06T04:30:35.874 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-10-06T04:30:35.901 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T04:30:35.991 INFO:teuthology.orchestra.run.smithi123.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-10-06T04:30:35.993 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-10-06T04:30:36.020 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T04:30:36.072 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-10-06T04:30:36.074 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-10-06T04:30:36.101 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T04:30:36.191 INFO:teuthology.orchestra.run.smithi177.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-10-06T04:30:36.193 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-10-06T04:30:36.220 INFO:tasks.ceph_client:Setting up client nodes... 2023-10-06T04:30:36.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-10-06T04:30:36.261 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-10-06T04:30:36.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-10-06T04:30:36.353 INFO:teuthology.orchestra.run.smithi123.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-10-06T04:30:36.366 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-10-06T04:30:36.407 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-10-06T04:30:36.419 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-10-06T04:30:36.500 INFO:teuthology.orchestra.run.smithi177.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-10-06T04:30:36.512 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-10-06T04:30:36.512 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi123.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi177.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-10-06T04:30:36.513 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-10-06T04:30:36.540 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:36.540 INFO:tasks.ceph:role: osd.0 2023-10-06T04:30:36.540 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:30:36.540 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:36.615 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:36.616 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:36.616 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:37.052 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-10-06T04:30:37.053 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-10-06T04:30:37.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-10-06T04:30:37.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-10-06T04:30:37.263 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-10-06T04:30:37.330 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:37.330 INFO:tasks.ceph:role: osd.1 2023-10-06T04:30:37.330 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:30:37.330 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T04:30:37.404 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:37.404 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:37.404 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:37.404 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:37.404 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:37.404 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:37.404 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:37.405 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:37.405 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:37.405 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:37.838 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-10-06T04:30:37.840 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-10-06T04:30:37.840 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-10-06T04:30:37.955 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-10-06T04:30:37.988 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-10-06T04:30:38.057 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:38.057 INFO:tasks.ceph:role: osd.2 2023-10-06T04:30:38.057 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:30:38.057 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T04:30:38.133 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:38.134 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:38.134 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:38.134 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:38.134 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:38.134 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:38.134 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:38.134 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:38.135 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:38.135 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:38.594 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-10-06T04:30:38.596 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-10-06T04:30:38.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-10-06T04:30:38.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-10-06T04:30:38.816 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-10-06T04:30:38.888 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:38.888 INFO:tasks.ceph:role: osd.3 2023-10-06T04:30:38.888 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:30:38.889 DEBUG:teuthology.orchestra.run.smithi123:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:38.966 INFO:teuthology.orchestra.run.smithi123.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:38.967 INFO:teuthology.orchestra.run.smithi123.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:38.967 INFO:teuthology.orchestra.run.smithi123.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:39.403 INFO:teuthology.orchestra.run.smithi123.stdout:Discarding blocks...Done. 2023-10-06T04:30:39.405 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi123.front.sepia.ceph.com -o noatime 2023-10-06T04:30:39.405 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-10-06T04:30:39.554 DEBUG:teuthology.orchestra.run.smithi123:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-10-06T04:30:39.595 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:39.696+0000 7fc928497540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-10-06T04:30:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:39.697+0000 7fc928497540 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-10-06T04:30:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:39.697+0000 7fc928497540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T04:30:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:39.697+0000 7fc928497540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T04:30:39.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:39.697+0000 7fc928497540 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-10-06T04:30:39.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:39.697+0000 7fc928497540 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-10-06T04:30:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:40.223+0000 7fc928497540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:40.988+0000 7fc928497540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:41.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:41.750+0000 7fc928497540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:42.545 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-10-06T04:30:42.579 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:42.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:42.671+0000 7f63bfaa2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-10-06T04:30:42.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:42.671+0000 7f63bfaa2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-10-06T04:30:42.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:42.671+0000 7f63bfaa2540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T04:30:42.674 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:42.671+0000 7f63bfaa2540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T04:30:42.674 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:42.671+0000 7f63bfaa2540 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-10-06T04:30:42.674 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:42.672+0000 7f63bfaa2540 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-10-06T04:30:43.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:43.197+0000 7f63bfaa2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:43.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:43.961+0000 7f63bfaa2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:44.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:44.723+0000 7f63bfaa2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:45.522 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-10-06T04:30:45.556 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:45.648 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:45.646+0000 7f8466a1b540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-10-06T04:30:45.648 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:45.647+0000 7f8466a1b540 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-10-06T04:30:45.648 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:45.647+0000 7f8466a1b540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T04:30:45.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:45.647+0000 7f8466a1b540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T04:30:45.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:45.647+0000 7f8466a1b540 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-10-06T04:30:45.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:45.647+0000 7f8466a1b540 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-10-06T04:30:46.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:46.175+0000 7f8466a1b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:46.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:46.941+0000 7f8466a1b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:47.707+0000 7f8466a1b540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:48.503 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-10-06T04:30:48.537 DEBUG:teuthology.orchestra.run.smithi123:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:48.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:48.628+0000 7f9f33295540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-10-06T04:30:48.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:48.629+0000 7f9f33295540 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-10-06T04:30:48.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:48.629+0000 7f9f33295540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T04:30:48.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:48.629+0000 7f9f33295540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T04:30:48.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:48.629+0000 7f9f33295540 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-10-06T04:30:48.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:48.629+0000 7f9f33295540 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-10-06T04:30:49.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:49.155+0000 7f9f33295540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:49.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:49.915+0000 7f9f33295540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:50.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:30:50.677+0000 7f9f33295540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:51.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-10-06T04:30:51.508 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-10-06T04:30:51.541 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:51.542 INFO:tasks.ceph:role: osd.4 2023-10-06T04:30:51.542 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:30:51.542 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-10-06T04:30:51.621 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:51.621 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:51.621 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:51.622 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:51.622 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:51.622 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:51.622 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:51.622 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:51.622 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:51.622 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:52.061 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T04:30:52.063 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T04:30:52.063 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-10-06T04:30:52.225 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-10-06T04:30:52.266 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-10-06T04:30:52.338 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:52.338 INFO:tasks.ceph:role: osd.5 2023-10-06T04:30:52.339 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:30:52.339 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-10-06T04:30:52.416 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:52.417 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:52.832 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T04:30:52.834 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T04:30:52.834 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-10-06T04:30:52.943 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-10-06T04:30:52.977 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-10-06T04:30:53.048 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:53.048 INFO:tasks.ceph:role: osd.6 2023-10-06T04:30:53.048 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:30:53.049 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-10-06T04:30:53.125 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:53.125 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:53.126 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:53.568 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T04:30:53.569 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T04:30:53.569 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-10-06T04:30:53.732 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-10-06T04:30:53.768 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-10-06T04:30:53.838 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-10-06T04:30:53.838 INFO:tasks.ceph:role: osd.7 2023-10-06T04:30:53.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:30:53.839 DEBUG:teuthology.orchestra.run.smithi177:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 attr=2, projid32bit=1 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout: = sunit=0 swidth=0 blks 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-10-06T04:30:53.916 INFO:teuthology.orchestra.run.smithi177.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-10-06T04:30:54.357 INFO:teuthology.orchestra.run.smithi177.stdout:Discarding blocks...Done. 2023-10-06T04:30:54.358 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi177.front.sepia.ceph.com -o noatime 2023-10-06T04:30:54.359 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-10-06T04:30:54.507 DEBUG:teuthology.orchestra.run.smithi177:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-10-06T04:30:54.540 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:54.644 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:54.641+0000 7fe8308b2540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-10-06T04:30:54.644 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:54.641+0000 7fe8308b2540 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-10-06T04:30:54.644 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:54.641+0000 7fe8308b2540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T04:30:54.644 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:54.641+0000 7fe8308b2540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T04:30:54.644 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:54.641+0000 7fe8308b2540 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-10-06T04:30:54.644 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:54.642+0000 7fe8308b2540 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-10-06T04:30:55.168 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:55.166+0000 7fe8308b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:55.932 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:55.930+0000 7fe8308b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:56.695 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:56.693+0000 7fe8308b2540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:57.492 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-10-06T04:30:57.525 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:30:57.613 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:57.611+0000 7f746a537540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-10-06T04:30:57.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:57.611+0000 7f746a537540 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-10-06T04:30:57.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:57.611+0000 7f746a537540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T04:30:57.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:57.611+0000 7f746a537540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T04:30:57.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:57.611+0000 7f746a537540 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-10-06T04:30:57.614 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:57.612+0000 7f746a537540 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-10-06T04:30:58.140 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:58.138+0000 7f746a537540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:58.902 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:58.900+0000 7f746a537540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:30:59.664 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:30:59.662+0000 7f746a537540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:31:00.461 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-10-06T04:31:00.494 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:31:00.582 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:00.580+0000 7f3f55278540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-10-06T04:31:00.582 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:00.580+0000 7f3f55278540 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-10-06T04:31:00.583 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:00.580+0000 7f3f55278540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T04:31:00.583 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:00.580+0000 7f3f55278540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T04:31:00.583 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:00.580+0000 7f3f55278540 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-10-06T04:31:00.583 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:00.581+0000 7f3f55278540 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-10-06T04:31:01.109 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:01.107+0000 7f3f55278540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:31:01.874 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:01.872+0000 7f3f55278540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:31:02.636 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:02.634+0000 7f3f55278540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:31:03.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:03.261 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:03.437 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-10-06T04:31:03.470 DEBUG:teuthology.orchestra.run.smithi177:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:31:03.561 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:03.559+0000 7ffa85211540 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-10-06T04:31:03.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:03.559+0000 7ffa85211540 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-10-06T04:31:03.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:03.559+0000 7ffa85211540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T04:31:03.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:03.559+0000 7ffa85211540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T04:31:03.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:03.559+0000 7ffa85211540 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-10-06T04:31:03.562 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:03.560+0000 7ffa85211540 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-10-06T04:31:04.087 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:04.085+0000 7ffa85211540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:31:04.851 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:04.849+0000 7ffa85211540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:31:05.618 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:05.616+0000 7ffa85211540 -1 bluestore::NCB::__restore_allocator::Failed open_for_read with error-code -2 2023-10-06T04:31:06.414 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-10-06T04:31:06.446 INFO:tasks.ceph:Reading keys from all nodes... 2023-10-06T04:31:06.446 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:06.446 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-10-06T04:31:06.480 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:06.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-10-06T04:31:06.551 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:06.551 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-10-06T04:31:06.622 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:06.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-10-06T04:31:06.691 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:06.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-10-06T04:31:06.759 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:06.760 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-10-06T04:31:06.828 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:06.828 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-10-06T04:31:06.896 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:06.896 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-10-06T04:31:06.929 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:06.929 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-10-06T04:31:07.000 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:07.001 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-10-06T04:31:07.071 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:07.071 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-10-06T04:31:07.140 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:07.140 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-10-06T04:31:07.209 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:07.209 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-10-06T04:31:07.277 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:07.278 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-10-06T04:31:07.346 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:07.346 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-10-06T04:31:07.365 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:07.365 DEBUG:teuthology.orchestra.run.smithi123:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-10-06T04:31:07.423 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:07.423 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-10-06T04:31:07.442 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:07.442 DEBUG:teuthology.orchestra.run.smithi177:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-10-06T04:31:07.500 INFO:tasks.ceph:Adding keys to all mons... 2023-10-06T04:31:07.500 DEBUG:teuthology.orchestra.run.smithi123:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T04:31:07.502 DEBUG:teuthology.orchestra.run.smithi177:> sudo tee -a /etc/ceph/ceph.keyring 2023-10-06T04:31:07.543 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.y] 2023-10-06T04:31:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBrjR9llNTxLBAAxlUBcZl4y4U9h573EW2L5g== 2023-10-06T04:31:07.543 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-10-06T04:31:07.543 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBrjR9ln+DxMxAAUlHVQYS9oN2DiGRGYWQ3qg== 2023-10-06T04:31:07.544 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-10-06T04:31:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBrjR9l8lYJOxAA3cNOazP2iJLFQgSr92kLzA== 2023-10-06T04:31:07.544 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.0] 2023-10-06T04:31:07.544 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBvjR9l0WieKRAAsku6SpM5TRo7hMMZNL24ZA== 2023-10-06T04:31:07.544 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.1] 2023-10-06T04:31:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQByjR9lgYsMKBAAFEXHSp6wWEg/EDfA3OstaQ== 2023-10-06T04:31:07.545 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.2] 2023-10-06T04:31:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB1jR9l7T6WJhAA5PQEt56J46fTUUJGKsEvxw== 2023-10-06T04:31:07.545 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.3] 2023-10-06T04:31:07.545 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB4jR9lNwGBJRAAZif6CdUCJ3upDIINQ4Y0Lw== 2023-10-06T04:31:07.546 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr.x] 2023-10-06T04:31:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBrjR9lWKDSMBAAmtSOktWG9AchefQzbynbFw== 2023-10-06T04:31:07.547 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-10-06T04:31:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsjR9ljvRFBBAAk1yK46SI75Y+WxBrG+kPQQ== 2023-10-06T04:31:07.547 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-10-06T04:31:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsjR9lRotZCxAAqXZ8VxmtgnlDE2xbHxwvlQ== 2023-10-06T04:31:07.547 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.4] 2023-10-06T04:31:07.547 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQB+jR9lbiROJhAAJbUY2WVm0VtRp2OAatPqEA== 2023-10-06T04:31:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.5] 2023-10-06T04:31:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCBjR9lpNCFJBAAlrytO/eUYhhp4ZgL6jVrtA== 2023-10-06T04:31:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.6] 2023-10-06T04:31:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCEjR9ld1GoIhAAh5yLDoZ5i3yGfZ6KGp/sug== 2023-10-06T04:31:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:[osd.7] 2023-10-06T04:31:07.548 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQCHjR9lUpZoIRAAVD5W176MnYqzrAkZVYMVuQ== 2023-10-06T04:31:07.548 INFO:teuthology.orchestra.run.smithi123.stdout:[client.0] 2023-10-06T04:31:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsjR9laHCGDxAAVQanXVVoF5JlcwGfxMHDDQ== 2023-10-06T04:31:07.549 INFO:teuthology.orchestra.run.smithi123.stdout:[client.2] 2023-10-06T04:31:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsjR9l5/0JFRAAk6plb7Uw0yiJ2JRHEMSHqA== 2023-10-06T04:31:07.549 INFO:teuthology.orchestra.run.smithi123.stdout:[client.1] 2023-10-06T04:31:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsjR9lwuE7GBAA9Dxl3aHJegzlIbs9znobYQ== 2023-10-06T04:31:07.549 INFO:teuthology.orchestra.run.smithi123.stdout:[client.3] 2023-10-06T04:31:07.549 INFO:teuthology.orchestra.run.smithi123.stdout: key = AQBsjR9lLA3QHRAADHdP6xY/vy9W5bGELFNMoA== 2023-10-06T04:31:07.565 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.y] 2023-10-06T04:31:07.565 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBrjR9llNTxLBAAxlUBcZl4y4U9h573EW2L5g== 2023-10-06T04:31:07.565 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBrjR9ln+DxMxAAUlHVQYS9oN2DiGRGYWQ3qg== 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBrjR9l8lYJOxAA3cNOazP2iJLFQgSr92kLzA== 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.0] 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBvjR9l0WieKRAAsku6SpM5TRo7hMMZNL24ZA== 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.1] 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQByjR9lgYsMKBAAFEXHSp6wWEg/EDfA3OstaQ== 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.2] 2023-10-06T04:31:07.566 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB1jR9l7T6WJhAA5PQEt56J46fTUUJGKsEvxw== 2023-10-06T04:31:07.567 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.3] 2023-10-06T04:31:07.567 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB4jR9lNwGBJRAAZif6CdUCJ3upDIINQ4Y0Lw== 2023-10-06T04:31:07.567 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr.x] 2023-10-06T04:31:07.567 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBrjR9lWKDSMBAAmtSOktWG9AchefQzbynbFw== 2023-10-06T04:31:07.567 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-10-06T04:31:07.567 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBsjR9ljvRFBBAAk1yK46SI75Y+WxBrG+kPQQ== 2023-10-06T04:31:07.567 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-10-06T04:31:07.568 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBsjR9lRotZCxAAqXZ8VxmtgnlDE2xbHxwvlQ== 2023-10-06T04:31:07.568 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.4] 2023-10-06T04:31:07.568 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQB+jR9lbiROJhAAJbUY2WVm0VtRp2OAatPqEA== 2023-10-06T04:31:07.568 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.5] 2023-10-06T04:31:07.568 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCBjR9lpNCFJBAAlrytO/eUYhhp4ZgL6jVrtA== 2023-10-06T04:31:07.569 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.6] 2023-10-06T04:31:07.569 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCEjR9ld1GoIhAAh5yLDoZ5i3yGfZ6KGp/sug== 2023-10-06T04:31:07.569 INFO:teuthology.orchestra.run.smithi177.stdout:[osd.7] 2023-10-06T04:31:07.569 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQCHjR9lUpZoIRAAVD5W176MnYqzrAkZVYMVuQ== 2023-10-06T04:31:07.569 INFO:teuthology.orchestra.run.smithi177.stdout:[client.0] 2023-10-06T04:31:07.569 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBsjR9laHCGDxAAVQanXVVoF5JlcwGfxMHDDQ== 2023-10-06T04:31:07.570 INFO:teuthology.orchestra.run.smithi177.stdout:[client.2] 2023-10-06T04:31:07.570 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBsjR9l5/0JFRAAk6plb7Uw0yiJ2JRHEMSHqA== 2023-10-06T04:31:07.570 INFO:teuthology.orchestra.run.smithi177.stdout:[client.1] 2023-10-06T04:31:07.570 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBsjR9lwuE7GBAA9Dxl3aHJegzlIbs9znobYQ== 2023-10-06T04:31:07.570 INFO:teuthology.orchestra.run.smithi177.stdout:[client.3] 2023-10-06T04:31:07.570 INFO:teuthology.orchestra.run.smithi177.stdout: key = AQBsjR9lLA3QHRAADHdP6xY/vy9W5bGELFNMoA== 2023-10-06T04:31:07.572 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:31:07.594 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:31:07.662 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:07.664 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:07.717 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:07.719 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:07.771 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.774 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.824 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.826 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.876 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.879 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.929 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.931 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:07.981 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:31:07.983 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-10-06T04:31:08.033 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:08.035 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:08.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:08.090 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-10-06T04:31:08.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.145 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.195 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.198 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.252 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.302 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.305 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-10-06T04:31:08.355 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.358 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.410 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.412 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.465 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.467 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.520 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.523 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-10-06T04:31:08.575 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-10-06T04:31:08.575 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-10-06T04:31:08.602 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T04:31:08.695 DEBUG:teuthology.orchestra.run.smithi123:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-10-06T04:31:08.722 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-10-06T04:31:08.750 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T04:31:08.844 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-10-06T04:31:08.871 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-10-06T04:31:08.939 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-10-06T04:31:09.033 DEBUG:teuthology.orchestra.run.smithi177:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-10-06T04:31:09.060 DEBUG:teuthology.orchestra.run.smithi123:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:31:09.062 DEBUG:teuthology.orchestra.run.smithi177:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-10-06T04:31:09.118 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-10-06T04:31:09.119 INFO:tasks.ceph.mon.a:Restarting daemon 2023-10-06T04:31:09.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-10-06T04:31:09.121 INFO:tasks.ceph.mon.a:Started 2023-10-06T04:31:09.121 INFO:tasks.ceph.mon.b:Restarting daemon 2023-10-06T04:31:09.121 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-10-06T04:31:09.160 INFO:tasks.ceph.mon.b:Started 2023-10-06T04:31:09.160 INFO:tasks.ceph.mon.c:Restarting daemon 2023-10-06T04:31:09.160 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-10-06T04:31:09.162 INFO:tasks.ceph.mon.c:Started 2023-10-06T04:31:09.162 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-10-06T04:31:09.162 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-10-06T04:31:09.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-10-06T04:31:09.164 INFO:tasks.ceph.mgr.y:Started 2023-10-06T04:31:09.164 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-10-06T04:31:09.165 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-10-06T04:31:09.166 INFO:tasks.ceph.mgr.x:Started 2023-10-06T04:31:09.166 INFO:tasks.ceph:Setting crush tunables to default 2023-10-06T04:31:09.167 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd crush tunables default 2023-10-06T04:31:09.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.285+0000 7effed393700 1 Processor -- start 2023-10-06T04:31:09.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.285+0000 7effed393700 1 -- start start 2023-10-06T04:31:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effed393700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effed393700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 0x7effe80b19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effed393700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 0x7effe8159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effed393700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe80b4040 con 0x7effe80b1f20 2023-10-06T04:31:09.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effed393700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe80af960 con 0x7effe80b0f00 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effed393700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe80b07a0 con 0x7effe81572b0 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 0x7effe8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 0x7effe8159fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47080/0 (socket says 172.21.15.123:47080) 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe77fe700 1 -- 172.21.15.123:0/1955659936 learned_addr learned my addr 172.21.15.123:0/1955659936 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe77fe700 1 -- 172.21.15.123:0/1955659936 >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 msgr2=0x7effe8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe77fe700 1 -- 172.21.15.123:0/1955659936 >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 msgr2=0x7effe8159fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe77fe700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 0x7effe8159fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe6ffd700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe67fc700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 0x7effe80b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe77fe700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 0x7effe8159fb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:31:09.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe6ffd700 1 -- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 msgr2=0x7effe80af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-10-06T04:31:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe6ffd700 1 -- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 msgr2=0x7effe80af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T04:31:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe6ffd700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T04:31:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe6ffd700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:31:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe67fc700 1 -- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 msgr2=0x7effe80b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-10-06T04:31:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe67fc700 1 -- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 msgr2=0x7effe80b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-10-06T04:31:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe67fc700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 0x7effe80b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 ((1) Operation not permitted) 2023-10-06T04:31:09.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.286+0000 7effe67fc700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 0x7effe80b19e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe77fe700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.123:3300/0 conn(0x7effe80b1f20 0x7effe8159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe67fc700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 0x7effe80b19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe6ffd700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe77fe700 1 -- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 msgr2=0x7effe80af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe77fe700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe77fe700 1 -- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 msgr2=0x7effe80b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe77fe700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 0x7effe80b19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe77fe700 1 -- 172.21.15.123:0/1955659936 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe80b2360 con 0x7effe80b1f20 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe6ffd700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:09.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe77fe700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe80b1f20 0x7effe8159fb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7effe806ad40 tx=0x7effe00091c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec16bebae230975c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe5ffb700 1 -- 172.21.15.123:0/1955659936 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0007ba0 con 0x7effe80b1f20 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe5ffb700 1 -- 172.21.15.123:0/1955659936 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe000f040 con 0x7effe80b1f20 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.487+0000 7effe5ffb700 1 -- 172.21.15.123:0/1955659936 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0013430 con 0x7effe80b1f20 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 -- 172.21.15.123:0/1955659936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe80b1f20 msgr2=0x7effe8159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe80b1f20 0x7effe8159fb0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7effe806ad40 tx=0x7effe00091c0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 -- 172.21.15.123:0/1955659936 shutdown_connections 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe80b1f20 0x7effe8159fb0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 --2- 172.21.15.123:0/1955659936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe81572b0 0x7effe80af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 --2- 172.21.15.123:0/1955659936 >> v2:172.21.15.177:3300/0 conn(0x7effe80b0f00 0x7effe80b19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 -- 172.21.15.123:0/1955659936 >> 172.21.15.123:0/1955659936 conn(0x7effe80973b0 msgr2=0x7effe8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 -- 172.21.15.123:0/1955659936 shutdown_connections 2023-10-06T04:31:09.492 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.488+0000 7effed393700 1 -- 172.21.15.123:0/1955659936 wait complete. 2023-10-06T04:31:09.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.489+0000 7effed393700 1 Processor -- start 2023-10-06T04:31:09.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.489+0000 7effed393700 1 -- start start 2023-10-06T04:31:09.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effed393700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effe80b0f00 0x7effe80ac700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effed393700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe80b1f20 0x7effe8159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effed393700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe81572b0 0x7effe81555a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effed393700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe80aafe0 con 0x7effe81572b0 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effed393700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe815b480 con 0x7effe80b0f00 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effed393700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe815aff0 con 0x7effe80b1f20 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effe77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe81572b0 0x7effe81555a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effe77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe81572b0 0x7effe81555a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47106/0 (socket says 172.21.15.123:47106) 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effe6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effe80b0f00 0x7effe80ac700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effe77fe700 1 -- 172.21.15.123:0/3349945756 learned_addr learned my addr 172.21.15.123:0/3349945756 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:09.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.490+0000 7effe67fc700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe80b1f20 0x7effe8159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effe77fe700 1 -- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe80b1f20 msgr2=0x7effe8159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effe77fe700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe80b1f20 0x7effe8159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effe77fe700 1 -- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effe80b0f00 msgr2=0x7effe80ac700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effe77fe700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effe80b0f00 0x7effe80ac700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effe77fe700 1 -- 172.21.15.123:0/3349945756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe0009760 con 0x7effe81572b0 2023-10-06T04:31:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effe6ffd700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effe80b0f00 0x7effe80ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:09.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effe77fe700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe81572b0 0x7effe81555a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7effe0009650 tx=0x7effe0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effceffd700 1 -- 172.21.15.123:0/3349945756 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0013c40 con 0x7effe81572b0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effceffd700 1 -- 172.21.15.123:0/3349945756 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe0013da0 con 0x7effe81572b0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe815a3e0 con 0x7effe81572b0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effceffd700 1 -- 172.21.15.123:0/3349945756 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe0027650 con 0x7effe81572b0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe815a9a0 con 0x7effe81572b0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effceffd700 1 -- 172.21.15.123:0/3349945756 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7effe001a030 con 0x7effe81572b0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.491+0000 7effceffd700 1 -- 172.21.15.123:0/3349945756 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 721+0+0 (secure 0 0 0) 0x7effe0027e20 con 0x7effe81572b0 2023-10-06T04:31:09.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.492+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe815b130 con 0x7effe81572b0 2023-10-06T04:31:09.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.494+0000 7effceffd700 1 -- 172.21.15.123:0/3349945756 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7effe002c020 con 0x7effe81572b0 2023-10-06T04:31:09.566 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:09.564+0000 7fd68d5c31c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:31:09.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.569+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7effe80afb90 con 0x7effe81572b0 2023-10-06T04:31:09.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.571+0000 7effceffd700 1 -- 172.21.15.123:0/3349945756 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7effe0038090 con 0x7effe81572b0 2023-10-06T04:31:09.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe81572b0 msgr2=0x7effe81555a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:09.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe81572b0 0x7effe81555a0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7effe0009650 tx=0x7effe0016000 comp rx=0 tx=0).stop 2023-10-06T04:31:09.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 shutdown_connections 2023-10-06T04:31:09.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effe81572b0 0x7effe81555a0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effe80b0f00 0x7effe80ac700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 --2- 172.21.15.123:0/3349945756 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effe80b1f20 0x7effe8159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:09.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 >> 172.21.15.123:0/3349945756 conn(0x7effe80973b0 msgr2=0x7effe806b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:09.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 shutdown_connections 2023-10-06T04:31:09.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:09.572+0000 7effed393700 1 -- 172.21.15.123:0/3349945756 wait complete. 2023-10-06T04:31:09.576 INFO:teuthology.orchestra.run.smithi123.stderr:adjusted tunables profile to default 2023-10-06T04:31:09.578 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:09.576+0000 7fab376311c0 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-10-06T04:31:09.586 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-10-06T04:31:09.587 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:09.587 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-10-06T04:31:09.613 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:09.613 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-10-06T04:31:09.686 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:09.686 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-10-06T04:31:09.733 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:09.731+0000 7fd68d5c31c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:31:09.763 DEBUG:teuthology.orchestra.run.smithi123:> set -ex 2023-10-06T04:31:09.763 DEBUG:teuthology.orchestra.run.smithi123:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-10-06T04:31:09.764 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:09.762+0000 7fab376311c0 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-10-06T04:31:09.792 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:09.792 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-10-06T04:31:09.820 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:09.820 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-10-06T04:31:09.886 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:09.887 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-10-06T04:31:09.954 DEBUG:teuthology.orchestra.run.smithi177:> set -ex 2023-10-06T04:31:09.954 DEBUG:teuthology.orchestra.run.smithi177:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-10-06T04:31:10.020 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 6c52dd00-164f-43a0-9150-7dabdfa7fb5c 0 2023-10-06T04:31:10.069 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:10.067+0000 7fd68d5c31c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:31:10.120 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:10.118+0000 7fab376311c0 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-10-06T04:31:10.180 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.178+0000 7f3df5cdf700 1 Processor -- start 2023-10-06T04:31:10.180 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.178+0000 7f3df5cdf700 1 -- start start 2023-10-06T04:31:10.182 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3df5cdf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df00c8710 0x7f3df00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.182 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3df5cdf700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3df0068670 0x7f3df0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3df5cdf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3df0069aa0 con 0x7f3df00c8340 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3df5cdf700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df00aa500 con 0x7f3df0068670 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3df5cdf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df0069050 con 0x7f3df00c8710 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df00c8710 0x7f3df00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df00c8710 0x7f3df00c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:37516/0 (socket says 172.21.15.177:37516) 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 -- 172.21.15.177:0/3633530330 learned_addr learned my addr 172.21.15.177:0/3633530330 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 -- 172.21.15.177:0/3633530330 >> v2:172.21.15.123:3300/0 conn(0x7f3df0068670 msgr2=0x7f3df0068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 --2- 172.21.15.177:0/3633530330 >> v2:172.21.15.123:3300/0 conn(0x7f3df0068670 0x7f3df0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 -- 172.21.15.177:0/3633530330 >> v1:172.21.15.123:6789/0 conn(0x7f3df00c8340 legacy=0x7f3df00a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 -- 172.21.15.177:0/3633530330 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df006a340 con 0x7f3df00c8710 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3deeffd700 1 --2- 172.21.15.177:0/3633530330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df00c8710 0x7f3df00c8b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3de40099e0 tx=0x7f3de4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ffad31e6889d7a2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.179+0000 7f3dee7fc700 1 -- 172.21.15.177:0/3633530330 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400b7e0 con 0x7f3df00c8710 2023-10-06T04:31:10.183 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3dee7fc700 1 -- 172.21.15.177:0/3633530330 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de400b940 con 0x7f3df00c8710 2023-10-06T04:31:10.184 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3dee7fc700 1 -- 172.21.15.177:0/3633530330 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de400bce0 con 0x7f3df00c8710 2023-10-06T04:31:10.184 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 -- 172.21.15.177:0/3633530330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df00c8710 msgr2=0x7f3df00c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.185 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 --2- 172.21.15.177:0/3633530330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df00c8710 0x7f3df00c8b60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f3de40099e0 tx=0x7f3de4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:10.185 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 -- 172.21.15.177:0/3633530330 shutdown_connections 2023-10-06T04:31:10.185 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 --2- 172.21.15.177:0/3633530330 >> v2:172.21.15.123:3300/0 conn(0x7f3df0068670 0x7f3df0068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.186 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 --2- 172.21.15.177:0/3633530330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df00c8710 0x7f3df00c8b60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.186 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 -- 172.21.15.177:0/3633530330 >> 172.21.15.177:0/3633530330 conn(0x7f3df00c9ca0 msgr2=0x7f3df006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:10.186 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 -- 172.21.15.177:0/3633530330 shutdown_connections 2023-10-06T04:31:10.186 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.180+0000 7f3df5cdf700 1 -- 172.21.15.177:0/3633530330 wait complete. 2023-10-06T04:31:10.188 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.181+0000 7f3df5cdf700 1 Processor -- start 2023-10-06T04:31:10.188 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.181+0000 7f3df5cdf700 1 -- start start 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3df5cdf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df0068670 0x7f3df0156650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3df5cdf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00c8340 0x7f3df015b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3df5cdf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3df00c8710 0x7f3df015b600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3df5cdf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00c2f00 con 0x7f3df00c8340 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3df5cdf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df016d820 con 0x7f3df00c8710 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3df5cdf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df00c2c20 con 0x7f3df0068670 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3def7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df0068670 0x7f3df0156650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3deffff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3df00c8710 0x7f3df015b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3def7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df0068670 0x7f3df0156650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:37528/0 (socket says 172.21.15.177:37528) 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3def7fe700 1 -- 172.21.15.177:0/2467506354 learned_addr learned my addr 172.21.15.177:0/2467506354 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:10.189 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.182+0000 7f3deeffd700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00c8340 0x7f3df015b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3def7fe700 1 -- 172.21.15.177:0/2467506354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3df00c8710 msgr2=0x7f3df015b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3def7fe700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3df00c8710 0x7f3df015b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3def7fe700 1 -- 172.21.15.177:0/2467506354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00c8340 msgr2=0x7f3df015b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3def7fe700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00c8340 0x7f3df015b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3def7fe700 1 -- 172.21.15.177:0/2467506354 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de40094f0 con 0x7f3df0068670 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3def7fe700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df0068670 0x7f3df0156650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3de000b630 tx=0x7f3de00091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3dd77fe700 1 -- 172.21.15.177:0/2467506354 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de000f510 con 0x7f3df0068670 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3dd77fe700 1 -- 172.21.15.177:0/2467506354 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de000f690 con 0x7f3df0068670 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3dd77fe700 1 -- 172.21.15.177:0/2467506354 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de000fa50 con 0x7f3df0068670 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df016d1d0 con 0x7f3df0068670 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df0157110 con 0x7f3df0068670 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3dd77fe700 1 -- 172.21.15.177:0/2467506354 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f3de001a400 con 0x7f3df0068670 2023-10-06T04:31:10.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.183+0000 7f3dd77fe700 1 -- 172.21.15.177:0/2467506354 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 721+0+0 (secure 0 0 0) 0x7f3de0011750 con 0x7f3df0068670 2023-10-06T04:31:10.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.184+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df00c8040 con 0x7f3df0068670 2023-10-06T04:31:10.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.186+0000 7f3dd77fe700 1 -- 172.21.15.177:0/2467506354 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f3de000fe00 con 0x7f3df0068670 2023-10-06T04:31:10.269 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.267+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6c52dd00-164f-43a0-9150-7dabdfa7fb5c", "id": 0} v 0) v1 -- 0x7f3df00aa6f0 con 0x7f3df0068670 2023-10-06T04:31:10.272 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.270+0000 7f3dd77fe700 1 -- 172.21.15.177:0/2467506354 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6c52dd00-164f-43a0-9150-7dabdfa7fb5c", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f3de00290b0 con 0x7f3df0068670 2023-10-06T04:31:10.272 INFO:teuthology.orchestra.run.smithi177.stdout:0 2023-10-06T04:31:10.273 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df0068670 msgr2=0x7f3df0156650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df0068670 0x7f3df0156650 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f3de000b630 tx=0x7f3de00091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 shutdown_connections 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3df00c8710 0x7f3df015b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3df0068670 0x7f3df0156650 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 --2- 172.21.15.177:0/2467506354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3df00c8340 0x7f3df015b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 >> 172.21.15.177:0/2467506354 conn(0x7f3df00c9ca0 msgr2=0x7f3df00af630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 shutdown_connections 2023-10-06T04:31:10.274 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.271+0000 7f3df5cdf700 1 -- 172.21.15.177:0/2467506354 wait complete. 2023-10-06T04:31:10.285 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 4616ab25-fd5a-44ae-a474-ee1400447513 1 2023-10-06T04:31:10.445 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.443+0000 7f284f1e8700 1 Processor -- start 2023-10-06T04:31:10.445 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.443+0000 7f284f1e8700 1 -- start start 2023-10-06T04:31:10.446 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.444+0000 7f284f1e8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f28481597d0 0x7f2848068770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.446 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.444+0000 7f284f1e8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480ccd30 con 0x7f28480cab40 2023-10-06T04:31:10.446 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.444+0000 7f284f1e8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28480c9640 con 0x7f28480cc930 2023-10-06T04:31:10.446 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.444+0000 7f284f1e8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28480cd530 con 0x7f28481597d0 2023-10-06T04:31:10.447 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.444+0000 7f284d785700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f28480cc930 0x7f28480c9ad0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:38964/0 (socket says 172.21.15.177:38964) 2023-10-06T04:31:10.447 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.444+0000 7f284d785700 1 -- 172.21.15.177:0/399337480 learned_addr learned my addr 172.21.15.177:0/399337480 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T04:31:10.447 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.444+0000 7f284cf84700 1 --2- 172.21.15.177:0/399337480 >> v2:172.21.15.123:3300/0 conn(0x7f28481597d0 0x7f2848068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.447 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.445+0000 7f284cf84700 1 -- 172.21.15.177:0/399337480 >> v1:172.21.15.177:6789/0 conn(0x7f28480cc930 legacy=0x7f28480c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.448 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.445+0000 7f284cf84700 1 -- 172.21.15.177:0/399337480 >> v1:172.21.15.123:6789/0 conn(0x7f28480cab40 legacy=0x7f28480cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.448 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.445+0000 7f284cf84700 1 -- 172.21.15.177:0/399337480 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28480c9030 con 0x7f28481597d0 2023-10-06T04:31:10.448 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.445+0000 7f284cf84700 1 --2- 172.21.15.177:0/399337480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28481597d0 0x7f2848068770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f283c0098d0 tx=0x7f283c009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab18f9c8bd0c8a26 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.449 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f28477fe700 1 -- 172.21.15.177:0/399337480 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c009e70 con 0x7f28481597d0 2023-10-06T04:31:10.449 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f28477fe700 1 -- 172.21.15.177:0/399337480 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f283c005c10 con 0x7f28481597d0 2023-10-06T04:31:10.449 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f28477fe700 1 -- 172.21.15.177:0/399337480 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f283c00b650 con 0x7f28481597d0 2023-10-06T04:31:10.450 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f284f1e8700 1 -- 172.21.15.177:0/399337480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28481597d0 msgr2=0x7f2848068770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.450 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f284f1e8700 1 --2- 172.21.15.177:0/399337480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28481597d0 0x7f2848068770 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f283c0098d0 tx=0x7f283c009120 comp rx=0 tx=0).stop 2023-10-06T04:31:10.450 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f284f1e8700 1 -- 172.21.15.177:0/399337480 shutdown_connections 2023-10-06T04:31:10.450 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f284f1e8700 1 --2- 172.21.15.177:0/399337480 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28481597d0 0x7f2848068770 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.450 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f284f1e8700 1 -- 172.21.15.177:0/399337480 >> 172.21.15.177:0/399337480 conn(0x7f2848097450 msgr2=0x7f2848067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:10.450 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f284f1e8700 1 -- 172.21.15.177:0/399337480 shutdown_connections 2023-10-06T04:31:10.450 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.447+0000 7f284f1e8700 1 -- 172.21.15.177:0/399337480 wait complete. 2023-10-06T04:31:10.451 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.449+0000 7f284f1e8700 1 Processor -- start 2023-10-06T04:31:10.451 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.449+0000 7f284f1e8700 1 -- start start 2023-10-06T04:31:10.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284f1e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28480cab40 0x7f28480c3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284f1e8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28480cc930 0x7f28480c41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284f1e8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28481597d0 0x7f28480b35f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284f1e8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480c2c00 con 0x7f28480cab40 2023-10-06T04:31:10.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284f1e8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28480c1ff0 con 0x7f28481597d0 2023-10-06T04:31:10.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284f1e8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28480c22d0 con 0x7f28480cc930 2023-10-06T04:31:10.454 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284d785700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28481597d0 0x7f28480b35f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f2847fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28480cc930 0x7f28480c41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284d785700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28481597d0 0x7f28480b35f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:52396/0 (socket says 172.21.15.177:52396) 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284d785700 1 -- 172.21.15.177:0/1040127942 learned_addr learned my addr 172.21.15.177:0/1040127942 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f2847fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28480cc930 0x7f28480c41a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:37538/0 (socket says 172.21.15.177:37538) 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.450+0000 7f284cf84700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28480cab40 0x7f28480c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2847fff700 1 -- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28481597d0 msgr2=0x7f28480b35f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2847fff700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28481597d0 0x7f28480b35f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2847fff700 1 -- 172.21.15.177:0/1040127942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28480cab40 msgr2=0x7f28480c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2847fff700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28480cab40 0x7f28480c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.455 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2847fff700 1 -- 172.21.15.177:0/1040127942 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f283c0093c0 con 0x7f28480cc930 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f284d785700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28481597d0 0x7f28480b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f284cf84700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28480cab40 0x7f28480c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2847fff700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28480cc930 0x7f28480c41a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f283400c490 tx=0x7f2834007ec0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2844ff9700 1 -- 172.21.15.177:0/1040127942 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834011590 con 0x7f28480cc930 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2844ff9700 1 -- 172.21.15.177:0/1040127942 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2834011710 con 0x7f28480cc930 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28480c25b0 con 0x7f28480cc930 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2844ff9700 1 -- 172.21.15.177:0/1040127942 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2834011ad0 con 0x7f28480cc930 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28480c46e0 con 0x7f28480cc930 2023-10-06T04:31:10.456 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2844ff9700 1 -- 172.21.15.177:0/1040127942 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f283401c6c0 con 0x7f28480cc930 2023-10-06T04:31:10.457 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.451+0000 7f2844ff9700 1 -- 172.21.15.177:0/1040127942 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 836+0+0 (secure 0 0 0) 0x7f28340127e0 con 0x7f28480cc930 2023-10-06T04:31:10.457 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.452+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28480c4900 con 0x7f28480cc930 2023-10-06T04:31:10.457 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.454+0000 7f2844ff9700 1 -- 172.21.15.177:0/1040127942 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f2834017070 con 0x7f28480cc930 2023-10-06T04:31:10.538 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.535+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "4616ab25-fd5a-44ae-a474-ee1400447513", "id": 1} v 0) v1 -- 0x7f28480c2d70 con 0x7f28480cc930 2023-10-06T04:31:10.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.543+0000 7f2844ff9700 1 -- 172.21.15.177:0/1040127942 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "4616ab25-fd5a-44ae-a474-ee1400447513", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f28480ccd30 con 0x7f28480cc930 2023-10-06T04:31:10.545 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-10-06T04:31:10.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28480cc930 msgr2=0x7f28480c41a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28480cc930 0x7f28480c41a0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f283400c490 tx=0x7f2834007ec0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 shutdown_connections 2023-10-06T04:31:10.546 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f28480cab40 0x7f28480c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f28481597d0 0x7f28480b35f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 --2- 172.21.15.177:0/1040127942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28480cc930 0x7f28480c41a0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 >> 172.21.15.177:0/1040127942 conn(0x7f2848097450 msgr2=0x7f2848154550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:10.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 shutdown_connections 2023-10-06T04:31:10.547 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.544+0000 7f284f1e8700 1 -- 172.21.15.177:0/1040127942 wait complete. 2023-10-06T04:31:10.558 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 7e970543-3fd8-472b-a1fa-bba74051dc6d 2 2023-10-06T04:31:10.597 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:10.595+0000 7fd68d5c31c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:31:10.629 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:10.627+0000 7fab376311c0 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2023-10-06T04:31:10.677 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.675+0000 7fa690492700 1 Processor -- start 2023-10-06T04:31:10.678 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.675+0000 7fa690492700 1 -- start start 2023-10-06T04:31:10.680 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa690492700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6881597d0 0x7fa688068770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.680 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa690492700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa6880cc930 0x7fa6880cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.680 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa690492700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6880ccd30 con 0x7fa6880cab40 2023-10-06T04:31:10.680 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa690492700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6880cd530 con 0x7fa6881597d0 2023-10-06T04:31:10.680 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa690492700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6880cb500 con 0x7fa6880cc930 2023-10-06T04:31:10.681 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa68ea2f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa6880cc930 0x7fa6880cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.681 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa68da2d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fa6880cab40 0x7fa6880cc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:38970/0 (socket says 172.21.15.177:38970) 2023-10-06T04:31:10.681 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa68da2d700 1 -- 172.21.15.177:0/2307550525 learned_addr learned my addr 172.21.15.177:0/2307550525 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T04:31:10.681 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa68ea2f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa6880cc930 0x7fa6880cafc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:52404/0 (socket says 172.21.15.177:52404) 2023-10-06T04:31:10.681 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.676+0000 7fa68e22e700 1 --2- 172.21.15.177:0/2307550525 >> v2:172.21.15.123:3300/0 conn(0x7fa6881597d0 0x7fa688068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.681 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68ea2f700 1 -- 172.21.15.177:0/2307550525 >> v2:172.21.15.123:3300/0 conn(0x7fa6881597d0 msgr2=0x7fa688068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:10.681 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68ea2f700 1 --2- 172.21.15.177:0/2307550525 >> v2:172.21.15.123:3300/0 conn(0x7fa6881597d0 0x7fa688068770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68ea2f700 1 -- 172.21.15.177:0/2307550525 >> v1:172.21.15.177:6789/0 conn(0x7fa6880cab40 legacy=0x7fa6880cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68ea2f700 1 -- 172.21.15.177:0/2307550525 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6880c9640 con 0x7fa6880cc930 2023-10-06T04:31:10.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68d22c700 1 -- 172.21.15.177:0/2307550525 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2985723549 0 0) 0x7fa678001110 con 0x7fa6880cab40 2023-10-06T04:31:10.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68ea2f700 1 --2- 172.21.15.177:0/2307550525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa6880cafc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6880cbbe0 tx=0x7fa680009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=18e2593305133ceb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68d22c700 1 -- 172.21.15.177:0/2307550525 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68000e070 con 0x7fa6880cc930 2023-10-06T04:31:10.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68d22c700 1 -- 172.21.15.177:0/2307550525 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68000bb60 con 0x7fa6880cc930 2023-10-06T04:31:10.682 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.677+0000 7fa68d22c700 1 -- 172.21.15.177:0/2307550525 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa680012430 con 0x7fa6880cc930 2023-10-06T04:31:10.683 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 -- 172.21.15.177:0/2307550525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 msgr2=0x7fa6880cafc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.683 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 --2- 172.21.15.177:0/2307550525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa6880cafc0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fa6880cbbe0 tx=0x7fa680009780 comp rx=0 tx=0).stop 2023-10-06T04:31:10.683 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 -- 172.21.15.177:0/2307550525 shutdown_connections 2023-10-06T04:31:10.683 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 --2- 172.21.15.177:0/2307550525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa6880cafc0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.683 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 --2- 172.21.15.177:0/2307550525 >> v2:172.21.15.123:3300/0 conn(0x7fa6881597d0 0x7fa688068770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.683 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 -- 172.21.15.177:0/2307550525 >> 172.21.15.177:0/2307550525 conn(0x7fa688097450 msgr2=0x7fa688067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:10.683 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 -- 172.21.15.177:0/2307550525 shutdown_connections 2023-10-06T04:31:10.684 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.678+0000 7fa690492700 1 -- 172.21.15.177:0/2307550525 wait complete. 2023-10-06T04:31:10.687 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.679+0000 7fa690492700 1 Processor -- start 2023-10-06T04:31:10.687 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.679+0000 7fa690492700 1 -- start start 2023-10-06T04:31:10.687 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6880cab40 0x7fa6881526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.687 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa68806c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.687 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6881597d0 0x7fa68806cb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880c6c70 con 0x7fa6880cab40 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880b2710 con 0x7fa6880cc930 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6880b1b00 con 0x7fa6881597d0 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa68806c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa68806c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:52412/0 (socket says 172.21.15.177:52412) 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68ea2f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6881597d0 0x7fa68806cb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 -- 172.21.15.177:0/1266721670 learned_addr learned my addr 172.21.15.177:0/1266721670 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68e22e700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6880cab40 0x7fa6881526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 -- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6881597d0 msgr2=0x7fa68806cb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.688 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6881597d0 0x7fa68806cb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 -- 172.21.15.177:0/1266721670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6880cab40 msgr2=0x7fa6881526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6880cab40 0x7fa6881526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 -- 172.21.15.177:0/1266721670 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa680009a20 con 0x7fa6880cc930 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68ea2f700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6881597d0 0x7fa68806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa68da2d700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa68806c650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c5b0 tx=0x7fa678001070 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa67e7fc700 1 -- 172.21.15.177:0/1266721670 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678005b30 con 0x7fa6880cc930 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa67e7fc700 1 -- 172.21.15.177:0/1266721670 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa678005c90 con 0x7fa6880cc930 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6880b1de0 con 0x7fa6880cc930 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa67e7fc700 1 -- 172.21.15.177:0/1266721670 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa678007300 con 0x7fa6880cc930 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6880b23a0 con 0x7fa6880cc930 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.680+0000 7fa67e7fc700 1 -- 172.21.15.177:0/1266721670 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fa67801c440 con 0x7fa6880cc930 2023-10-06T04:31:10.689 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.681+0000 7fa67e7fc700 1 -- 172.21.15.177:0/1266721670 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 951+0+0 (secure 0 0 0) 0x7fa678005e00 con 0x7fa6880cc930 2023-10-06T04:31:10.690 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.681+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa68806d840 con 0x7fa6880cc930 2023-10-06T04:31:10.690 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.683+0000 7fa67e7fc700 1 -- 172.21.15.177:0/1266721670 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fa678017070 con 0x7fa6880cc930 2023-10-06T04:31:10.739 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:10.737+0000 7fd68d5c31c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:31:10.766 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:10.764+0000 7fab376311c0 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2023-10-06T04:31:10.768 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.767+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd new", "uuid": "7e970543-3fd8-472b-a1fa-bba74051dc6d", "id": 2} v 0) v1 -- 0x7fa6880b2850 con 0x7fa6880cc930 2023-10-06T04:31:10.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.774+0000 7fa67e7fc700 1 -- 172.21.15.177:0/1266721670 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "7e970543-3fd8-472b-a1fa-bba74051dc6d", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7fa6880ccd30 con 0x7fa6880cc930 2023-10-06T04:31:10.776 INFO:teuthology.orchestra.run.smithi177.stdout:2 2023-10-06T04:31:10.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 msgr2=0x7fa68806c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa68806c650 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fa67800c5b0 tx=0x7fa678001070 comp rx=0 tx=0).stop 2023-10-06T04:31:10.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 shutdown_connections 2023-10-06T04:31:10.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa6880cab40 0x7fa6881526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6881597d0 0x7fa68806cb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 --2- 172.21.15.177:0/1266721670 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6880cc930 0x7fa68806c650 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 >> 172.21.15.177:0/1266721670 conn(0x7fa688097450 msgr2=0x7fa688153a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:10.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 shutdown_connections 2023-10-06T04:31:10.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.775+0000 7fa690492700 1 -- 172.21.15.177:0/1266721670 wait complete. 2023-10-06T04:31:10.789 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 48b5e737-f098-4e53-b19f-cce0011cf3b5 3 2023-10-06T04:31:10.950 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.947+0000 7f89cc272700 1 Processor -- start 2023-10-06T04:31:10.950 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.947+0000 7f89cc272700 1 -- start start 2023-10-06T04:31:10.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.948+0000 7f89cc272700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f89c4157450 0x7f89c4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.948+0000 7f89cc272700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4068bb0 0x7f89c40c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.948+0000 7f89cc272700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89c40c7810 con 0x7f89c4157080 2023-10-06T04:31:10.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.948+0000 7f89cc272700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89c4069030 con 0x7f89c4157450 2023-10-06T04:31:10.952 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.948+0000 7f89cc272700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c40ac9f0 con 0x7f89c4068bb0 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca80f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4068bb0 0x7f89c40c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca00e700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f89c4157080 0x7f89c40abf30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:38972/0 (socket says 172.21.15.177:38972) 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca00e700 1 -- 172.21.15.177:0/2766631311 learned_addr learned my addr 172.21.15.177:0/2766631311 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89c980d700 1 --2- 172.21.15.177:0/2766631311 >> v2:172.21.15.123:3300/0 conn(0x7f89c4157450 0x7f89c4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca80f700 1 -- 172.21.15.177:0/2766631311 >> v2:172.21.15.123:3300/0 conn(0x7f89c4157450 msgr2=0x7f89c4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca80f700 1 --2- 172.21.15.177:0/2766631311 >> v2:172.21.15.123:3300/0 conn(0x7f89c4157450 0x7f89c4068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89c900c700 1 -- 172.21.15.177:0/2766631311 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2389352710 0 0) 0x7f89c40c7810 con 0x7f89c4157080 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca80f700 1 -- 172.21.15.177:0/2766631311 >> v1:172.21.15.177:6789/0 conn(0x7f89c4157080 legacy=0x7f89c40abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca80f700 1 -- 172.21.15.177:0/2766631311 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89c40c6ee0 con 0x7f89c4068bb0 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89ca80f700 1 --2- 172.21.15.177:0/2766631311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4068bb0 0x7f89c40c7ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009e60 tx=0x7f89bc009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=15a0d39d0cc6da38 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.953 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89c900c700 1 -- 172.21.15.177:0/2766631311 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc00e070 con 0x7f89c4068bb0 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89c900c700 1 -- 172.21.15.177:0/2766631311 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc00bb60 con 0x7f89c4068bb0 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.949+0000 7f89c900c700 1 -- 172.21.15.177:0/2766631311 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc012430 con 0x7f89c4068bb0 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 -- 172.21.15.177:0/2766631311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4068bb0 msgr2=0x7f89c40c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 --2- 172.21.15.177:0/2766631311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4068bb0 0x7f89c40c7ca0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f89bc009e60 tx=0x7f89bc009780 comp rx=0 tx=0).stop 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 -- 172.21.15.177:0/2766631311 shutdown_connections 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 --2- 172.21.15.177:0/2766631311 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4068bb0 0x7f89c40c7ca0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 --2- 172.21.15.177:0/2766631311 >> v2:172.21.15.123:3300/0 conn(0x7f89c4157450 0x7f89c4068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 -- 172.21.15.177:0/2766631311 >> 172.21.15.177:0/2766631311 conn(0x7f89c40c9ca0 msgr2=0x7f89c406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 -- 172.21.15.177:0/2766631311 shutdown_connections 2023-10-06T04:31:10.954 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.950+0000 7f89cc272700 1 -- 172.21.15.177:0/2766631311 wait complete. 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.951+0000 7f89cc272700 1 Processor -- start 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.951+0000 7f89cc272700 1 -- start start 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89cc272700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89c4068bb0 0x7f89c415a030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89cc272700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4157080 0x7f89c40c5b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89cc272700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4157450 0x7f89c415cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89cc272700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c406a370 con 0x7f89c4157080 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89cc272700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89c40c6620 con 0x7f89c4068bb0 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89cc272700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89c4069a40 con 0x7f89c4157450 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca00e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89c4068bb0 0x7f89c415a030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.957 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca80f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4157450 0x7f89c415cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca00e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89c4068bb0 0x7f89c415a030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:52422/0 (socket says 172.21.15.177:52422) 2023-10-06T04:31:10.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca00e700 1 -- 172.21.15.177:0/1833489158 learned_addr learned my addr 172.21.15.177:0/1833489158 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:10.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89c980d700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4157080 0x7f89c40c5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:10.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca80f700 1 -- 172.21.15.177:0/1833489158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89c4068bb0 msgr2=0x7f89c415a030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca80f700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89c4068bb0 0x7f89c415a030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca80f700 1 -- 172.21.15.177:0/1833489158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4157080 msgr2=0x7f89c40c5b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:10.958 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca80f700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4157080 0x7f89c40c5b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca80f700 1 -- 172.21.15.177:0/1833489158 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89bc009a20 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ca80f700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4157450 0x7f89c415cf50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89bc016000 tx=0x7f89bc019000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ba7fc700 1 -- 172.21.15.177:0/1833489158 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc012ea0 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ba7fc700 1 -- 172.21.15.177:0/1833489158 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89bc027470 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89ba7fc700 1 -- 172.21.15.177:0/1833489158 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89bc027830 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.952+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89c4069d20 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.953+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89c415a570 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.953+0000 7f89ba7fc700 1 -- 172.21.15.177:0/1833489158 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f89bc024070 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.953+0000 7f89ba7fc700 1 -- 172.21.15.177:0/1833489158 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1066+0+0 (secure 0 0 0) 0x7f89bc01da50 con 0x7f89c4157450 2023-10-06T04:31:10.959 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.954+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89c40c71c0 con 0x7f89c4157450 2023-10-06T04:31:10.960 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:10.956+0000 7f89ba7fc700 1 -- 172.21.15.177:0/1833489158 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f89bc0476d0 con 0x7f89c4157450 2023-10-06T04:31:11.041 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.038+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "48b5e737-f098-4e53-b19f-cce0011cf3b5", "id": 3} v 0) v1 -- 0x7f89c404bb80 con 0x7f89c4157450 2023-10-06T04:31:11.048 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.046+0000 7f89ba7fc700 1 -- 172.21.15.177:0/1833489158 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "48b5e737-f098-4e53-b19f-cce0011cf3b5", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f89bc036030 con 0x7f89c4157450 2023-10-06T04:31:11.048 INFO:teuthology.orchestra.run.smithi177.stdout:3 2023-10-06T04:31:11.049 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4157450 msgr2=0x7f89c415cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.049 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4157450 0x7f89c415cf50 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f89bc016000 tx=0x7f89bc019000 comp rx=0 tx=0).stop 2023-10-06T04:31:11.049 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 shutdown_connections 2023-10-06T04:31:11.049 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89c4157450 0x7f89c415cf50 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.049 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89c4068bb0 0x7f89c415a030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.049 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 --2- 172.21.15.177:0/1833489158 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89c4157080 0x7f89c40c5b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.049 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 >> 172.21.15.177:0/1833489158 conn(0x7f89c40c9ca0 msgr2=0x7f89c40cb9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:11.050 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 shutdown_connections 2023-10-06T04:31:11.050 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.047+0000 7f89cc272700 1 -- 172.21.15.177:0/1833489158 wait complete. 2023-10-06T04:31:11.064 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 013a1be3-b369-47e4-9dd9-93a620898e3a 4 2023-10-06T04:31:11.099 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:11.097+0000 7fab376311c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:31:11.105 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:11.103+0000 7fd68d5c31c0 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-10-06T04:31:11.186 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.184+0000 7f7541911700 1 Processor -- start 2023-10-06T04:31:11.186 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.184+0000 7f7541911700 1 -- start start 2023-10-06T04:31:11.190 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f7541911700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f753c0b20f0 0x7f753c0af420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f7541911700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c159fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f7541911700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f753c0af960 con 0x7f753c1572b0 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f7541911700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f753c0b4040 con 0x7f753c0b20f0 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f7541911700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0b07a0 con 0x7f753c0b0f00 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753affd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f753c1572b0 0x7f753c0b19e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:38974/0 (socket says 172.21.15.177:38974) 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753affd700 1 -- 172.21.15.177:0/450087729 learned_addr learned my addr 172.21.15.177:0/450087729 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753b7fe700 1 --2- 172.21.15.177:0/450087729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c159fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753a7fc700 1 --2- 172.21.15.177:0/450087729 >> v2:172.21.15.123:3300/0 conn(0x7f753c0b20f0 0x7f753c0af420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f7539ffb700 1 -- 172.21.15.177:0/450087729 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 555280358 0 0) 0x7f753c0af960 con 0x7f753c1572b0 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f7539ffb700 1 -- 172.21.15.177:0/450087729 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7524003570 con 0x7f753c1572b0 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753b7fe700 1 -- 172.21.15.177:0/450087729 >> v2:172.21.15.123:3300/0 conn(0x7f753c0b20f0 msgr2=0x7f753c0af420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:11.191 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753b7fe700 1 --2- 172.21.15.177:0/450087729 >> v2:172.21.15.123:3300/0 conn(0x7f753c0b20f0 0x7f753c0af420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753b7fe700 1 -- 172.21.15.177:0/450087729 >> v1:172.21.15.177:6789/0 conn(0x7f753c1572b0 legacy=0x7f753c0b19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753b7fe700 1 -- 172.21.15.177:0/450087729 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f753c0afb90 con 0x7f753c0b0f00 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.185+0000 7f753a7fc700 1 --2- 172.21.15.177:0/450087729 >> v2:172.21.15.123:3300/0 conn(0x7f753c0b20f0 0x7f753c0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f753b7fe700 1 --2- 172.21.15.177:0/450087729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c159fb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7534009e60 tx=0x7f7534009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=80cf0aca2e60cc0f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7539ffb700 1 -- 172.21.15.177:0/450087729 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f753400e070 con 0x7f753c0b0f00 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7539ffb700 1 -- 172.21.15.177:0/450087729 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f753400bb60 con 0x7f753c0b0f00 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7539ffb700 1 -- 172.21.15.177:0/450087729 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7534012430 con 0x7f753c0b0f00 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7541911700 1 -- 172.21.15.177:0/450087729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 msgr2=0x7f753c159fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7541911700 1 --2- 172.21.15.177:0/450087729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c159fb0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f7534009e60 tx=0x7f7534009780 comp rx=0 tx=0).stop 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7541911700 1 -- 172.21.15.177:0/450087729 shutdown_connections 2023-10-06T04:31:11.192 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7541911700 1 --2- 172.21.15.177:0/450087729 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c159fb0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.193 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7541911700 1 --2- 172.21.15.177:0/450087729 >> v2:172.21.15.123:3300/0 conn(0x7f753c0b20f0 0x7f753c0af420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.193 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7541911700 1 -- 172.21.15.177:0/450087729 >> 172.21.15.177:0/450087729 conn(0x7f753c0973b0 msgr2=0x7f753c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:11.193 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.186+0000 7f7541911700 1 -- 172.21.15.177:0/450087729 shutdown_connections 2023-10-06T04:31:11.193 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.187+0000 7f7541911700 1 -- 172.21.15.177:0/450087729 wait complete. 2023-10-06T04:31:11.193 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.188+0000 7f7541911700 1 Processor -- start 2023-10-06T04:31:11.193 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.188+0000 7f7541911700 1 -- start start 2023-10-06T04:31:11.195 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.188+0000 7f7541911700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c148510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.196 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.188+0000 7f7541911700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f753c0b20f0 0x7f753c0a8fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.196 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7541911700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f753c1572b0 0x7f753c15ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.196 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7541911700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c0aabd0 con 0x7f753c1572b0 2023-10-06T04:31:11.196 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7541911700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f753c16d660 con 0x7f753c0b20f0 2023-10-06T04:31:11.196 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7541911700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f753c16ca50 con 0x7f753c0b0f00 2023-10-06T04:31:11.196 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c148510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753a7fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f753c0b20f0 0x7f753c0a8fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c148510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:37574/0 (socket says 172.21.15.177:37574) 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 -- 172.21.15.177:0/4146839424 learned_addr learned my addr 172.21.15.177:0/4146839424 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753b7fe700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f753c1572b0 0x7f753c15ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 -- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f753c0b20f0 msgr2=0x7f753c0a8fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f753c0b20f0 0x7f753c0a8fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 -- 172.21.15.177:0/4146839424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f753c1572b0 msgr2=0x7f753c15ba90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f753c1572b0 0x7f753c15ba90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 -- 172.21.15.177:0/4146839424 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7534009a20 con 0x7f753c0b0f00 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753a7fc700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f753c0b20f0 0x7f753c0a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f753affd700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c148510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f753000c790 tx=0x7f7530007c80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:11.197 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7522ffd700 1 -- 172.21.15.177:0/4146839424 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75300072e0 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7522ffd700 1 -- 172.21.15.177:0/4146839424 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7530007440 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7522ffd700 1 -- 172.21.15.177:0/4146839424 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75300057f0 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f753c16cd30 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.189+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f753c16d2f0 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.190+0000 7f7522ffd700 1 -- 172.21.15.177:0/4146839424 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f753001c440 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.190+0000 7f7522ffd700 1 -- 172.21.15.177:0/4146839424 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1181+0+0 (secure 0 0 0) 0x7f7530012910 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.190+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f753c16d4b0 con 0x7f753c0b0f00 2023-10-06T04:31:11.198 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.192+0000 7f7522ffd700 1 -- 172.21.15.177:0/4146839424 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f7530021060 con 0x7f753c0b0f00 2023-10-06T04:31:11.357 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.355+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "013a1be3-b369-47e4-9dd9-93a620898e3a", "id": 4} v 0) v1 -- 0x7f753c0afe70 con 0x7f753c0b0f00 2023-10-06T04:31:11.364 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.362+0000 7f7522ffd700 1 -- 172.21.15.177:0/4146839424 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "013a1be3-b369-47e4-9dd9-93a620898e3a", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7530017070 con 0x7f753c0b0f00 2023-10-06T04:31:11.365 INFO:teuthology.orchestra.run.smithi177.stdout:4 2023-10-06T04:31:11.366 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 msgr2=0x7f753c148510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.366 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c148510 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f753000c790 tx=0x7f7530007c80 comp rx=0 tx=0).stop 2023-10-06T04:31:11.366 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 shutdown_connections 2023-10-06T04:31:11.367 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f753c1572b0 0x7f753c15ba90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.367 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f753c0b0f00 0x7f753c148510 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.367 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 --2- 172.21.15.177:0/4146839424 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f753c0b20f0 0x7f753c0a8fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.367 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 >> 172.21.15.177:0/4146839424 conn(0x7f753c0973b0 msgr2=0x7f753c09ae40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:11.367 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 shutdown_connections 2023-10-06T04:31:11.367 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.364+0000 7f7541911700 1 -- 172.21.15.177:0/4146839424 wait complete. 2023-10-06T04:31:11.381 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new 2b7e928e-e086-4c2a-9d76-9ba1b050044c 5 2023-10-06T04:31:11.539 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.537+0000 7f37dc980700 1 Processor -- start 2023-10-06T04:31:11.540 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.537+0000 7f37dc980700 1 -- start start 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37dc980700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f37d813fa70 0x7f37d80c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37dc980700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d806e9f0 0x7f37d80c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37dc980700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d806a4e0 con 0x7f37d80cabf0 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37dc980700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d80c8ec0 con 0x7f37d813fa70 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37dc980700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80c91a0 con 0x7f37d806e9f0 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d6d9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d806e9f0 0x7f37d80c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d5d9b700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f37d80cabf0 0x7f37d806e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:38988/0 (socket says 172.21.15.177:38988) 2023-10-06T04:31:11.543 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d5d9b700 1 -- 172.21.15.177:0/56214767 learned_addr learned my addr 172.21.15.177:0/56214767 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d659c700 1 --2- 172.21.15.177:0/56214767 >> v2:172.21.15.123:3300/0 conn(0x7f37d813fa70 0x7f37d80c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d559a700 1 -- 172.21.15.177:0/56214767 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2219456448 0 0) 0x7f37d806a4e0 con 0x7f37d80cabf0 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d6d9d700 1 -- 172.21.15.177:0/56214767 >> v2:172.21.15.123:3300/0 conn(0x7f37d813fa70 msgr2=0x7f37d80c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d6d9d700 1 --2- 172.21.15.177:0/56214767 >> v2:172.21.15.123:3300/0 conn(0x7f37d813fa70 0x7f37d80c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d6d9d700 1 -- 172.21.15.177:0/56214767 >> v1:172.21.15.177:6789/0 conn(0x7f37d80cabf0 legacy=0x7f37d806e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d6d9d700 1 -- 172.21.15.177:0/56214767 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d80c9480 con 0x7f37d806e9f0 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.539+0000 7f37d6d9d700 1 --2- 172.21.15.177:0/56214767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d806e9f0 0x7f37d80c8840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009e60 tx=0x7f37d0009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2259d585e63baf9f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37d559a700 1 -- 172.21.15.177:0/56214767 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d000e070 con 0x7f37d806e9f0 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37d559a700 1 -- 172.21.15.177:0/56214767 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d000bb60 con 0x7f37d806e9f0 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37d559a700 1 -- 172.21.15.177:0/56214767 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d0012430 con 0x7f37d806e9f0 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 -- 172.21.15.177:0/56214767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d806e9f0 msgr2=0x7f37d80c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.544 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 --2- 172.21.15.177:0/56214767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d806e9f0 0x7f37d80c8840 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f37d0009e60 tx=0x7f37d0009780 comp rx=0 tx=0).stop 2023-10-06T04:31:11.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 -- 172.21.15.177:0/56214767 shutdown_connections 2023-10-06T04:31:11.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 --2- 172.21.15.177:0/56214767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d806e9f0 0x7f37d80c8840 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 --2- 172.21.15.177:0/56214767 >> v2:172.21.15.123:3300/0 conn(0x7f37d813fa70 0x7f37d80c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 -- 172.21.15.177:0/56214767 >> 172.21.15.177:0/56214767 conn(0x7f37d80cd020 msgr2=0x7f37d80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:11.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 -- 172.21.15.177:0/56214767 shutdown_connections 2023-10-06T04:31:11.545 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.540+0000 7f37dc980700 1 -- 172.21.15.177:0/56214767 wait complete. 2023-10-06T04:31:11.548 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.542+0000 7f37dc980700 1 Processor -- start 2023-10-06T04:31:11.548 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.542+0000 7f37dc980700 1 -- start start 2023-10-06T04:31:11.548 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37dc980700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37d806e9f0 0x7f37d81555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.548 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37dc980700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d80cabf0 0x7f37d8134570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.548 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37dc980700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37d813fa70 0x7f37d8134ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.548 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37dc980700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80b2450 con 0x7f37d813fa70 2023-10-06T04:31:11.548 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37dc980700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80b26f0 con 0x7f37d806e9f0 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37dc980700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d80aeda0 con 0x7f37d80cabf0 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d80cabf0 0x7f37d8134570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d80cabf0 0x7f37d8134570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.177:37592/0 (socket says 172.21.15.177:37592) 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 -- 172.21.15.177:0/662036008 learned_addr learned my addr 172.21.15.177:0/662036008 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d6d9d700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37d813fa70 0x7f37d8134ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d659c700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37d806e9f0 0x7f37d81555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 -- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37d806e9f0 msgr2=0x7f37d81555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37d806e9f0 0x7f37d81555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 -- 172.21.15.177:0/662036008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37d813fa70 msgr2=0x7f37d8134ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.549 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37d813fa70 0x7f37d8134ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 -- 172.21.15.177:0/662036008 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d0009a20 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d6d9d700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37d813fa70 0x7f37d8134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d659c700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37d806e9f0 0x7f37d81555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.543+0000 7f37d5d9b700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d80cabf0 0x7f37d8134570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c7e0 tx=0x7f37cc007df0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.544+0000 7f37c67fc700 1 -- 172.21.15.177:0/662036008 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc005b30 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.544+0000 7f37c67fc700 1 -- 172.21.15.177:0/662036008 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37cc005c90 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.544+0000 7f37c67fc700 1 -- 172.21.15.177:0/662036008 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37cc007300 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.544+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37d80af080 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.544+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d80b2b80 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.544+0000 7f37c67fc700 1 -- 172.21.15.177:0/662036008 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f37cc01c440 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.544+0000 7f37c67fc700 1 -- 172.21.15.177:0/662036008 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1296+0+0 (secure 0 0 0) 0x7f37cc005e00 con 0x7f37d80cabf0 2023-10-06T04:31:11.550 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.545+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d806d240 con 0x7f37d80cabf0 2023-10-06T04:31:11.551 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.547+0000 7f37c67fc700 1 -- 172.21.15.177:0/662036008 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f37cc017070 con 0x7f37d80cabf0 2023-10-06T04:31:11.629 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.627+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "2b7e928e-e086-4c2a-9d76-9ba1b050044c", "id": 5} v 0) v1 -- 0x7f37d80b2830 con 0x7f37d80cabf0 2023-10-06T04:31:11.637 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.635+0000 7f37c67fc700 1 -- 172.21.15.177:0/662036008 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "2b7e928e-e086-4c2a-9d76-9ba1b050044c", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f37cc01c8d0 con 0x7f37d80cabf0 2023-10-06T04:31:11.637 INFO:teuthology.orchestra.run.smithi177.stdout:5 2023-10-06T04:31:11.638 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d80cabf0 msgr2=0x7f37d8134570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.638 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d80cabf0 0x7f37d8134570 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f37cc00c7e0 tx=0x7f37cc007df0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.638 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 shutdown_connections 2023-10-06T04:31:11.638 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37d813fa70 0x7f37d8134ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.638 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37d806e9f0 0x7f37d81555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.638 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 --2- 172.21.15.177:0/662036008 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37d80cabf0 0x7f37d8134570 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.639 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 >> 172.21.15.177:0/662036008 conn(0x7f37d80cd020 msgr2=0x7f37d80c9900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:11.639 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 shutdown_connections 2023-10-06T04:31:11.639 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.636+0000 7f37dc980700 1 -- 172.21.15.177:0/662036008 wait complete. 2023-10-06T04:31:11.649 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new c42e4b64-fcb7-4e6f-b071-bd16175cd367 6 2023-10-06T04:31:11.768 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.766+0000 7f14ad491700 1 Processor -- start 2023-10-06T04:31:11.768 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.766+0000 7f14ad491700 1 -- start start 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14ad491700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f14a813f9f0 0x7f14a806b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14ad491700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a806b020 con 0x7f14a813fdc0 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14ad491700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14a806a450 con 0x7f14a806b9f0 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14ad491700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14a806a6f0 con 0x7f14a813f9f0 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a77fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f14a806b9f0 0x7f14a80c7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:38996/0 (socket says 172.21.15.177:38996) 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a77fe700 1 -- 172.21.15.177:0/613982706 learned_addr learned my addr 172.21.15.177:0/613982706 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a6ffd700 1 --2- 172.21.15.177:0/613982706 >> v2:172.21.15.123:3300/0 conn(0x7f14a813f9f0 0x7f14a806b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a5ffb700 1 -- 172.21.15.177:0/613982706 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2136533464 0 0) 0x7f14a806a450 con 0x7f14a806b9f0 2023-10-06T04:31:11.770 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a5ffb700 1 -- 172.21.15.177:0/613982706 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1490003570 con 0x7f14a806b9f0 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a5ffb700 1 -- 172.21.15.177:0/613982706 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2220833906 0 0) 0x7f14a806b020 con 0x7f14a813fdc0 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a5ffb700 1 -- 172.21.15.177:0/613982706 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a806a450 con 0x7f14a813fdc0 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a6ffd700 1 -- 172.21.15.177:0/613982706 >> v1:172.21.15.177:6789/0 conn(0x7f14a806b9f0 legacy=0x7f14a80c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a6ffd700 1 -- 172.21.15.177:0/613982706 >> v1:172.21.15.123:6789/0 conn(0x7f14a813fdc0 legacy=0x7f14a8067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a6ffd700 1 -- 172.21.15.177:0/613982706 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14a806a9d0 con 0x7f14a813f9f0 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.767+0000 7f14a6ffd700 1 --2- 172.21.15.177:0/613982706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a806b4b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14980098e0 tx=0x7f14980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89e9f95e983fc9d3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14a5ffb700 1 -- 172.21.15.177:0/613982706 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498011630 con 0x7f14a813f9f0 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14a5ffb700 1 -- 172.21.15.177:0/613982706 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1498011790 con 0x7f14a813f9f0 2023-10-06T04:31:11.771 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14a5ffb700 1 -- 172.21.15.177:0/613982706 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1498011b30 con 0x7f14a813f9f0 2023-10-06T04:31:11.774 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14ad491700 1 -- 172.21.15.177:0/613982706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 msgr2=0x7f14a806b4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.774 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14ad491700 1 --2- 172.21.15.177:0/613982706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a806b4b0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f14980098e0 tx=0x7f14980091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.774 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14ad491700 1 -- 172.21.15.177:0/613982706 shutdown_connections 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14ad491700 1 --2- 172.21.15.177:0/613982706 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a806b4b0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14ad491700 1 -- 172.21.15.177:0/613982706 >> 172.21.15.177:0/613982706 conn(0x7f14a80cd020 msgr2=0x7f14a80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14ad491700 1 -- 172.21.15.177:0/613982706 shutdown_connections 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.768+0000 7f14ad491700 1 -- 172.21.15.177:0/613982706 wait complete. 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 Processor -- start 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 -- start start 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14a806b9f0 0x7f14a815a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a8142b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14a81401a0 0x7f14a8140640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8141ab0 con 0x7f14a813f9f0 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14a8169960 con 0x7f14a806b9f0 2023-10-06T04:31:11.775 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14ad491700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14a816ccf0 con 0x7f14a81401a0 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14a6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14a806b9f0 0x7f14a815a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14a77fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14a81401a0 0x7f14a8140640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14a6ffd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14a806b9f0 0x7f14a815a630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:52438/0 (socket says 172.21.15.177:52438) 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.770+0000 7f14a6ffd700 1 -- 172.21.15.177:0/4033184136 learned_addr learned my addr 172.21.15.177:0/4033184136 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a67fc700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a8142b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a77fe700 1 -- 172.21.15.177:0/4033184136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14a806b9f0 msgr2=0x7f14a815a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a77fe700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14a806b9f0 0x7f14a815a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a77fe700 1 -- 172.21.15.177:0/4033184136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 msgr2=0x7f14a8142b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a77fe700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a8142b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a77fe700 1 -- 172.21.15.177:0/4033184136 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1498009420 con 0x7f14a81401a0 2023-10-06T04:31:11.776 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a67fc700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a8142b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14a77fe700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14a81401a0 0x7f14a8140640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f14a0001460 tx=0x7f14a0001a40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f148effd700 1 -- 172.21.15.177:0/4033184136 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000a840 con 0x7f14a81401a0 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f148effd700 1 -- 172.21.15.177:0/4033184136 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14a000a9a0 con 0x7f14a81401a0 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f148effd700 1 -- 172.21.15.177:0/4033184136 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14a000ad60 con 0x7f14a81401a0 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14a8142710 con 0x7f14a81401a0 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14a81414f0 con 0x7f14a81401a0 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f148effd700 1 -- 172.21.15.177:0/4033184136 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f14a0007300 con 0x7f14a81401a0 2023-10-06T04:31:11.777 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.771+0000 7f148effd700 1 -- 172.21.15.177:0/4033184136 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1411+0+0 (secure 0 0 0) 0x7f14a001ba00 con 0x7f14a81401a0 2023-10-06T04:31:11.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.773+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14a806d240 con 0x7f14a81401a0 2023-10-06T04:31:11.778 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.775+0000 7f148effd700 1 -- 172.21.15.177:0/4033184136 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f14a8142710 con 0x7f14a81401a0 2023-10-06T04:31:11.815 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:11.813+0000 7fab376311c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:31:11.860 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.858+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "c42e4b64-fcb7-4e6f-b071-bd16175cd367", "id": 6} v 0) v1 -- 0x7f14a813bcb0 con 0x7f14a81401a0 2023-10-06T04:31:11.868 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.866+0000 7f148effd700 1 -- 172.21.15.177:0/4033184136 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "c42e4b64-fcb7-4e6f-b071-bd16175cd367", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f14a002a0b0 con 0x7f14a81401a0 2023-10-06T04:31:11.868 INFO:teuthology.orchestra.run.smithi177.stdout:6 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14a81401a0 msgr2=0x7f14a8140640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14a81401a0 0x7f14a8140640 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f14a0001460 tx=0x7f14a0001a40 comp rx=0 tx=0).stop 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 shutdown_connections 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f14a81401a0 0x7f14a8140640 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f14a806b9f0 0x7f14a815a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 --2- 172.21.15.177:0/4033184136 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f14a813f9f0 0x7f14a8142b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 >> 172.21.15.177:0/4033184136 conn(0x7f14a80cd020 msgr2=0x7f14a8067660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:11.869 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 shutdown_connections 2023-10-06T04:31:11.870 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.867+0000 7f14ad491700 1 -- 172.21.15.177:0/4033184136 wait complete. 2023-10-06T04:31:11.880 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --cluster ceph osd new b4976457-ab03-4387-b58e-0ac8bf0dd489 7 2023-10-06T04:31:11.885 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:11.883+0000 7fab376311c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:31:11.931 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:11.929+0000 7fd68d5c31c0 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-10-06T04:31:11.998 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.996+0000 7f8186641700 1 Processor -- start 2023-10-06T04:31:11.999 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.996+0000 7f8186641700 1 -- start start 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f8186641700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f818013fa70 0x7f81800c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f8186641700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f81800cabf0 0x7f81800c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f8186641700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f818006a4e0 con 0x7f818006e2e0 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f8186641700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800c8ec0 con 0x7f818013fa70 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f8186641700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81800c91a0 con 0x7f81800cabf0 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f817f7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f81800cabf0 0x7f81800c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f8184bde700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f818006e2e0 0x7f818006a970 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.177:39002/0 (socket says 172.21.15.177:39002) 2023-10-06T04:31:12.001 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.997+0000 7f8184bde700 1 -- 172.21.15.177:0/3916099835 learned_addr learned my addr 172.21.15.177:0/3916099835 (peer_addr_for_me v1:172.21.15.177:0/0) 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817ffff700 1 --2- 172.21.15.177:0/3916099835 >> v2:172.21.15.123:3300/0 conn(0x7f818013fa70 0x7f81800c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817f7fe700 1 -- 172.21.15.177:0/3916099835 >> v2:172.21.15.123:3300/0 conn(0x7f818013fa70 msgr2=0x7f81800c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817f7fe700 1 --2- 172.21.15.177:0/3916099835 >> v2:172.21.15.123:3300/0 conn(0x7f818013fa70 0x7f81800c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817f7fe700 1 -- 172.21.15.177:0/3916099835 >> v1:172.21.15.177:6789/0 conn(0x7f818006e2e0 legacy=0x7f818006a970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817effd700 1 -- 172.21.15.177:0/3916099835 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3506723727 0 0) 0x7f818006a4e0 con 0x7f818006e2e0 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817f7fe700 1 -- 172.21.15.177:0/3916099835 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81800c9480 con 0x7f81800cabf0 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817f7fe700 1 --2- 172.21.15.177:0/3916099835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800c8840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f81800b3020 tx=0x7f81740091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ef5aa220912d468c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817effd700 1 -- 172.21.15.177:0/3916099835 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011630 con 0x7f81800cabf0 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817effd700 1 -- 172.21.15.177:0/3916099835 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8174011790 con 0x7f81800cabf0 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.998+0000 7f817effd700 1 -- 172.21.15.177:0/3916099835 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8174011b30 con 0x7f81800cabf0 2023-10-06T04:31:12.002 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 -- 172.21.15.177:0/3916099835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 msgr2=0x7f81800c8840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 --2- 172.21.15.177:0/3916099835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800c8840 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f81800b3020 tx=0x7f81740091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 -- 172.21.15.177:0/3916099835 shutdown_connections 2023-10-06T04:31:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 --2- 172.21.15.177:0/3916099835 >> v2:172.21.15.123:3300/0 conn(0x7f818013fa70 0x7f81800c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 --2- 172.21.15.177:0/3916099835 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800c8840 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 -- 172.21.15.177:0/3916099835 >> 172.21.15.177:0/3916099835 conn(0x7f81800cd020 msgr2=0x7f81800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 -- 172.21.15.177:0/3916099835 shutdown_connections 2023-10-06T04:31:12.003 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:11.999+0000 7f8186641700 1 -- 172.21.15.177:0/3916099835 wait complete. 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.000+0000 7f8186641700 1 Processor -- start 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.001+0000 7f8186641700 1 -- start start 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.001+0000 7f8186641700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818006e2e0 0x7f81800b22f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.001+0000 7f8186641700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800cc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.001+0000 7f8186641700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818013fa70 0x7f8180165c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.001+0000 7f8186641700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8180166980 con 0x7f818013fa70 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.001+0000 7f8186641700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81800b1550 con 0x7f81800cabf0 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.001+0000 7f8186641700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81800b1830 con 0x7f818006e2e0 2023-10-06T04:31:12.006 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817f7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800cc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817ffff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818006e2e0 0x7f81800b22f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817f7fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800cc0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.177:52460/0 (socket says 172.21.15.177:52460) 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817f7fe700 1 -- 172.21.15.177:0/657721620 learned_addr learned my addr 172.21.15.177:0/657721620 (peer_addr_for_me v2:172.21.15.177:0/0) 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f8184bde700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818013fa70 0x7f8180165c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817ffff700 1 -- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 msgr2=0x7f81800cc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817ffff700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800cc0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817ffff700 1 -- 172.21.15.177:0/657721620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818013fa70 msgr2=0x7f8180165c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817ffff700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818013fa70 0x7f8180165c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817ffff700 1 -- 172.21.15.177:0/657721620 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8174009420 con 0x7f818006e2e0 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f8184bde700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818013fa70 0x7f8180165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817f7fe700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:12.007 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f817ffff700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818006e2e0 0x7f81800b22f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f817000e900 tx=0x7f81700091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f8167fff700 1 -- 172.21.15.177:0/657721620 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170013680 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f8167fff700 1 -- 172.21.15.177:0/657721620 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81700137e0 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81800b1b10 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f8167fff700 1 -- 172.21.15.177:0/657721620 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8170013ba0 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.002+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81800b2830 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.003+0000 7f8167fff700 1 -- 172.21.15.177:0/657721620 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f8170005c80 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.003+0000 7f8167fff700 1 -- 172.21.15.177:0/657721620 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1526+0+0 (secure 0 0 0) 0x7f8170006570 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.003+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f818006d240 con 0x7f818006e2e0 2023-10-06T04:31:12.008 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.003+0000 7fd68d5c31c0 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-10-06T04:31:12.009 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.005+0000 7f8167fff700 1 -- 172.21.15.177:0/657721620 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f8170021060 con 0x7f818006e2e0 2023-10-06T04:31:12.018 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.016+0000 7fab376311c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:31:12.085 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.083+0000 7fab376311c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:31:12.090 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.088+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b4976457-ab03-4387-b58e-0ac8bf0dd489", "id": 7} v 0) v1 -- 0x7f818004bb80 con 0x7f818006e2e0 2023-10-06T04:31:12.093 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.092+0000 7f8167fff700 1 -- 172.21.15.177:0/657721620 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b4976457-ab03-4387-b58e-0ac8bf0dd489", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7f8170017070 con 0x7f818006e2e0 2023-10-06T04:31:12.094 INFO:teuthology.orchestra.run.smithi177.stdout:7 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818006e2e0 msgr2=0x7f81800b22f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818006e2e0 0x7f81800b22f0 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f817000e900 tx=0x7f81700091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 shutdown_connections 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f818013fa70 0x7f8180165c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f818006e2e0 0x7f81800b22f0 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 --2- 172.21.15.177:0/657721620 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81800cabf0 0x7f81800cc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 >> 172.21.15.177:0/657721620 conn(0x7f81800cd020 msgr2=0x7f818013ed70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:12.095 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 shutdown_connections 2023-10-06T04:31:12.096 INFO:teuthology.orchestra.run.smithi177.stderr:2023-10-06T04:31:12.093+0000 7f8186641700 1 -- 172.21.15.177:0/657721620 wait complete. 2023-10-06T04:31:12.106 INFO:tasks.ceph.osd.0:Restarting daemon 2023-10-06T04:31:12.107 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-10-06T04:31:12.109 INFO:tasks.ceph.osd.0:Started 2023-10-06T04:31:12.109 INFO:tasks.ceph.osd.1:Restarting daemon 2023-10-06T04:31:12.109 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-10-06T04:31:12.111 INFO:tasks.ceph.osd.1:Started 2023-10-06T04:31:12.111 INFO:tasks.ceph.osd.2:Restarting daemon 2023-10-06T04:31:12.111 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-10-06T04:31:12.113 INFO:tasks.ceph.osd.2:Started 2023-10-06T04:31:12.113 INFO:tasks.ceph.osd.3:Restarting daemon 2023-10-06T04:31:12.113 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-10-06T04:31:12.115 INFO:tasks.ceph.osd.3:Started 2023-10-06T04:31:12.115 INFO:tasks.ceph.osd.4:Restarting daemon 2023-10-06T04:31:12.115 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-10-06T04:31:12.117 INFO:tasks.ceph.osd.4:Started 2023-10-06T04:31:12.117 INFO:tasks.ceph.osd.5:Restarting daemon 2023-10-06T04:31:12.117 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-10-06T04:31:12.119 INFO:tasks.ceph.osd.5:Started 2023-10-06T04:31:12.119 INFO:tasks.ceph.osd.6:Restarting daemon 2023-10-06T04:31:12.119 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-10-06T04:31:12.121 INFO:tasks.ceph.osd.6:Started 2023-10-06T04:31:12.122 INFO:tasks.ceph.osd.7:Restarting daemon 2023-10-06T04:31:12.122 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-10-06T04:31:12.125 INFO:tasks.ceph.osd.7:Started 2023-10-06T04:31:12.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:12.154 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.152+0000 7fd68d5c31c0 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-10-06T04:31:12.233 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.231+0000 7fd68d5c31c0 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-10-06T04:31:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.274+0000 7fc0f4b20700 1 Processor -- start 2023-10-06T04:31:12.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.274+0000 7fc0f4b20700 1 -- start start 2023-10-06T04:31:12.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0f4b20700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc0f00b05b0 0x7fc0f00b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0f4b20700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc0f00b0a20 0x7fc0f00b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0f4b20700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0f015b370 con 0x7fc0f01591b0 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0f4b20700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f0159c90 con 0x7fc0f00b05b0 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0f4b20700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0f015b0d0 con 0x7fc0f00b0a20 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc0f00b05b0 0x7fc0f00b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc0f00b05b0 0x7fc0f00b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47586/0 (socket says 172.21.15.123:47586) 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 -- 172.21.15.123:0/2807842384 learned_addr learned my addr 172.21.15.123:0/2807842384 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0eed9d700 1 --2- 172.21.15.123:0/2807842384 >> v2:172.21.15.177:3300/0 conn(0x7fc0f00b0a20 0x7fc0f00b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 -- 172.21.15.123:0/2807842384 >> v2:172.21.15.177:3300/0 conn(0x7fc0f00b0a20 msgr2=0x7fc0f00b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:12.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 --2- 172.21.15.123:0/2807842384 >> v2:172.21.15.177:3300/0 conn(0x7fc0f00b0a20 0x7fc0f00b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 -- 172.21.15.123:0/2807842384 >> v1:172.21.15.177:6789/0 conn(0x7fc0f01591b0 legacy=0x7fc0f0159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 -- 172.21.15.123:0/2807842384 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0f00b0fe0 con 0x7fc0f00b05b0 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0edd9b700 1 --2- 172.21.15.123:0/2807842384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f00b05b0 0x7fc0f00b3ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00b5530 tx=0x7fc0e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e1c91089a798ddfb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0ed59a700 1 -- 172.21.15.123:0/2807842384 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400b860 con 0x7fc0f00b05b0 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0ed59a700 1 -- 172.21.15.123:0/2807842384 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e400b9c0 con 0x7fc0f00b05b0 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.275+0000 7fc0ed59a700 1 -- 172.21.15.123:0/2807842384 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e400bd60 con 0x7fc0f00b05b0 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2807842384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f00b05b0 msgr2=0x7fc0f00b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 --2- 172.21.15.123:0/2807842384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f00b05b0 0x7fc0f00b3ca0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fc0f00b5530 tx=0x7fc0e4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2807842384 shutdown_connections 2023-10-06T04:31:12.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 --2- 172.21.15.123:0/2807842384 >> v2:172.21.15.177:3300/0 conn(0x7fc0f00b0a20 0x7fc0f00b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 --2- 172.21.15.123:0/2807842384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f00b05b0 0x7fc0f00b3ca0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2807842384 >> 172.21.15.123:0/2807842384 conn(0x7fc0f00995a0 msgr2=0x7fc0f0063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:12.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2807842384 shutdown_connections 2023-10-06T04:31:12.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.276+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2807842384 wait complete. 2023-10-06T04:31:12.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.277+0000 7fc0f4b20700 1 Processor -- start 2023-10-06T04:31:12.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.277+0000 7fc0f4b20700 1 -- start start 2023-10-06T04:31:12.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0f4b20700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f00b05b0 0x7fc0f0156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0f4b20700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f00b0a20 0x7fc0f016ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0f4b20700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f01591b0 0x7fc0f00ae310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:12.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0f4b20700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f01574a0 con 0x7fc0f01591b0 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0f4b20700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f009c5f0 con 0x7fc0f00b0a20 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0f4b20700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0f009bce0 con 0x7fc0f00b05b0 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f01591b0 0x7fc0f00ae310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f01591b0 0x7fc0f00ae310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47590/0 (socket says 172.21.15.123:47590) 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 -- 172.21.15.123:0/2031418501 learned_addr learned my addr 172.21.15.123:0/2031418501 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0edd9b700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f00b0a20 0x7fc0f016ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0ee59c700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f00b05b0 0x7fc0f0156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 -- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f00b05b0 msgr2=0x7fc0f0156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f00b05b0 0x7fc0f0156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 -- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f00b0a20 msgr2=0x7fc0f016ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f00b0a20 0x7fc0f016ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.278+0000 7fc0eed9d700 1 -- 172.21.15.123:0/2031418501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0e40094f0 con 0x7fc0f01591b0 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0eed9d700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f01591b0 0x7fc0f00ae310 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009fd0 tx=0x7fc0e8009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0de7fc700 1 -- 172.21.15.123:0/2031418501 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e800ebf0 con 0x7fc0f01591b0 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0de7fc700 1 -- 172.21.15.123:0/2031418501 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0e800ed50 con 0x7fc0f01591b0 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0f009bfa0 con 0x7fc0f01591b0 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0de7fc700 1 -- 172.21.15.123:0/2031418501 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0e8014670 con 0x7fc0f01591b0 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0f016b290 con 0x7fc0f01591b0 2023-10-06T04:31:12.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0de7fc700 1 -- 172.21.15.123:0/2031418501 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7fc0e80149e0 con 0x7fc0f01591b0 2023-10-06T04:31:12.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.279+0000 7fc0de7fc700 1 -- 172.21.15.123:0/2031418501 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7fc0e8014c90 con 0x7fc0f01591b0 2023-10-06T04:31:12.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.280+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0f00688f0 con 0x7fc0f01591b0 2023-10-06T04:31:12.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.282+0000 7fc0de7fc700 1 -- 172.21.15.123:0/2031418501 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7fc0e80273e0 con 0x7fc0f01591b0 2023-10-06T04:31:12.298 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.295+0000 7fab376311c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:31:12.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.349+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc0f009c730 con 0x7fc0f01591b0 2023-10-06T04:31:12.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.350+0000 7fc0de7fc700 1 -- 172.21.15.123:0/2031418501 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fc0e8027580 con 0x7fc0f01591b0 2023-10-06T04:31:12.352 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:12.353 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":10,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:12.090395+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:12.353 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f01591b0 msgr2=0x7fc0f00ae310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f01591b0 0x7fc0f00ae310 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fc0e8009fd0 tx=0x7fc0e8009900 comp rx=0 tx=0).stop 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 shutdown_connections 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc0f01591b0 0x7fc0f00ae310 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0f00b05b0 0x7fc0f0156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 --2- 172.21.15.123:0/2031418501 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc0f00b0a20 0x7fc0f016ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 >> 172.21.15.123:0/2031418501 conn(0x7fc0f00995a0 msgr2=0x7fc0f00681e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 shutdown_connections 2023-10-06T04:31:12.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:12.351+0000 7fc0f4b20700 1 -- 172.21.15.123:0/2031418501 wait complete. 2023-10-06T04:31:12.365 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-10-06T04:31:12.365 INFO:tasks.ceph:Waiting for OSDs to come up 2023-10-06T04:31:12.377 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.374+0000 7fab376311c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:31:12.414 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.412+0000 7fd68d5c31c0 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-10-06T04:31:12.452 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.450+0000 7fab376311c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:31:12.491 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.489+0000 7fd68d5c31c0 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-10-06T04:31:12.543 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.541+0000 7fab376311c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:31:12.567 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.565+0000 7fd68d5c31c0 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-10-06T04:31:12.663 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.661+0000 7fd68d5c31c0 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-10-06T04:31:12.719 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T04:31:12.717+0000 7fe6c2ace540 -1 Falling back to public interface 2023-10-06T04:31:12.720 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.717+0000 7fab376311c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:31:12.724 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T04:31:12.722+0000 7f7706942540 -1 Falling back to public interface 2023-10-06T04:31:12.735 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T04:31:12.733+0000 7f37f888c540 -1 Falling back to public interface 2023-10-06T04:31:12.739 INFO:tasks.ceph.osd.3.smithi123.stderr:2023-10-06T04:31:12.737+0000 7f62169d3540 -1 Falling back to public interface 2023-10-06T04:31:12.740 INFO:tasks.ceph.osd.0.smithi123.stderr:2023-10-06T04:31:12.738+0000 7f640f304540 -1 Falling back to public interface 2023-10-06T04:31:12.743 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T04:31:12.741+0000 7ff5fbc80540 -1 Falling back to public interface 2023-10-06T04:31:12.745 INFO:tasks.ceph.osd.1.smithi123.stderr:2023-10-06T04:31:12.743+0000 7fd099854540 -1 Falling back to public interface 2023-10-06T04:31:12.749 INFO:tasks.ceph.osd.2.smithi123.stderr:2023-10-06T04:31:12.747+0000 7ff067196540 -1 Falling back to public interface 2023-10-06T04:31:12.804 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:12.801+0000 7fab376311c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:31:12.843 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.841+0000 7fd68d5c31c0 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-10-06T04:31:12.922 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:12.920+0000 7fd68d5c31c0 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-10-06T04:31:13.086 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:13.084+0000 7fab376311c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:31:13.155 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:13.152+0000 7fab376311c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:31:13.170 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:13.209 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:13.207+0000 7fd68d5c31c0 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-10-06T04:31:13.277 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:13.275+0000 7fd68d5c31c0 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-10-06T04:31:13.280 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.278+0000 7f280b4f0700 1 Processor -- start 2023-10-06T04:31:13.281 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.278+0000 7f280b4f0700 1 -- start start 2023-10-06T04:31:13.283 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f280b4f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f280406e960 0x7f28040c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:13.283 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f280b4f0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f280406f080 0x7f28040ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:13.283 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f280b4f0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28040c8580 con 0x7f28041560c0 2023-10-06T04:31:13.283 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f280b4f0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28040c8860 con 0x7f280406f080 2023-10-06T04:31:13.283 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f280b4f0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28040c8b40 con 0x7f280406e960 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f2809a8d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f280406f080 0x7f28040ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f280928c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f28041560c0 0x7f280406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:37510/0 (socket says 172.21.15.123:37510) 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f280928c700 1 -- 172.21.15.123:0/2509396822 learned_addr learned my addr 172.21.15.123:0/2509396822 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.279+0000 7f2809a8d700 1 --2- 172.21.15.123:0/2509396822 >> v2:172.21.15.123:3300/0 conn(0x7f280406f080 0x7f28040ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47606/0 (socket says 172.21.15.123:47606) 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f2808a8b700 1 --2- 172.21.15.123:0/2509396822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f280406e960 0x7f28040c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f2809a8d700 1 -- 172.21.15.123:0/2509396822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f280406e960 msgr2=0x7f28040c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f2809a8d700 1 --2- 172.21.15.123:0/2509396822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f280406e960 0x7f28040c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f2809a8d700 1 -- 172.21.15.123:0/2509396822 >> v1:172.21.15.123:6789/0 conn(0x7f28041560c0 legacy=0x7f280406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f2809a8d700 1 -- 172.21.15.123:0/2509396822 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28040c8e20 con 0x7f280406f080 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f27fbfff700 1 -- 172.21.15.123:0/2509396822 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619356765 0 0) 0x7f28040c8580 con 0x7f28041560c0 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f2808a8b700 1 --2- 172.21.15.123:0/2509396822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f280406e960 0x7f28040c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:13.284 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f2809a8d700 1 --2- 172.21.15.123:0/2509396822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406f080 0x7f28040ada60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009e60 tx=0x7f27fc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3556741aa86a281 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f27fbfff700 1 -- 172.21.15.123:0/2509396822 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc00e070 con 0x7f280406f080 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f27fbfff700 1 -- 172.21.15.123:0/2509396822 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27fc00bc30 con 0x7f280406f080 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.280+0000 7f27fbfff700 1 -- 172.21.15.123:0/2509396822 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27fc012430 con 0x7f280406f080 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 -- 172.21.15.123:0/2509396822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406f080 msgr2=0x7f28040ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 --2- 172.21.15.123:0/2509396822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406f080 0x7f28040ada60 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f27fc009e60 tx=0x7f27fc009780 comp rx=0 tx=0).stop 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 -- 172.21.15.123:0/2509396822 shutdown_connections 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 --2- 172.21.15.123:0/2509396822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406f080 0x7f28040ada60 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 --2- 172.21.15.123:0/2509396822 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f280406e960 0x7f28040c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 -- 172.21.15.123:0/2509396822 >> 172.21.15.123:0/2509396822 conn(0x7f28040cda80 msgr2=0x7f28040c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 -- 172.21.15.123:0/2509396822 shutdown_connections 2023-10-06T04:31:13.285 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.281+0000 7f280b4f0700 1 -- 172.21.15.123:0/2509396822 wait complete. 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.282+0000 7f280b4f0700 1 Processor -- start 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.282+0000 7f280b4f0700 1 -- start start 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280b4f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406e960 0x7f28041553e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280b4f0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f280406f080 0x7f280415a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280b4f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28041560c0 0x7f28041679b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280b4f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280415b2d0 con 0x7f280406e960 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280b4f0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f280415b5b0 con 0x7f280406f080 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280b4f0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f280415b890 con 0x7f28041560c0 2023-10-06T04:31:13.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406e960 0x7f28041553e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406e960 0x7f28041553e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:47612/0 (socket says 172.21.15.123:47612) 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 -- 172.21.15.123:0/1864923372 learned_addr learned my addr 172.21.15.123:0/1864923372 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f2809a8d700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28041560c0 0x7f28041679b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f2808a8b700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f280406f080 0x7f280415a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 -- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28041560c0 msgr2=0x7f28041679b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28041560c0 0x7f28041679b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 -- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f280406f080 msgr2=0x7f280415a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f280406f080 0x7f280415a7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 -- 172.21.15.123:0/1864923372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27fc009a20 con 0x7f280406e960 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f2809a8d700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28041560c0 0x7f28041679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.283+0000 7f280928c700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406e960 0x7f28041553e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c5b0 tx=0x7f27f4001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:13.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.284+0000 7f27f97fa700 1 -- 172.21.15.123:0/1864923372 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4005b30 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.284+0000 7f27f97fa700 1 -- 172.21.15.123:0/1864923372 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27f4005c90 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.284+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f280415bb70 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.284+0000 7f27f97fa700 1 -- 172.21.15.123:0/1864923372 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27f4007300 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.284+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2804155920 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.284+0000 7f27f97fa700 1 -- 172.21.15.123:0/1864923372 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 1) v1 ==== 521+0+0 (secure 0 0 0) 0x7f27f401c6c0 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.284+0000 7f27f97fa700 1 -- 172.21.15.123:0/1864923372 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1641+0+0 (secure 0 0 0) 0x7f27f4005e00 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.285+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27e40052b0 con 0x7f280406e960 2023-10-06T04:31:13.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.287+0000 7f27f97fa700 1 -- 172.21.15.123:0/1864923372 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+71349 (secure 0 0 0) 0x7f27f4017070 con 0x7f280406e960 2023-10-06T04:31:13.358 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.355+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27e4005830 con 0x7f280406e960 2023-10-06T04:31:13.358 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.356+0000 7f27f97fa700 1 -- 172.21.15.123:0/1864923372 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7013 (secure 0 0 0) 0x7f27f401cb50 con 0x7f280406e960 2023-10-06T04:31:13.358 INFO:teuthology.misc.health.smithi123.stdout: 2023-10-06T04:31:13.359 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":10,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:12.090395+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:13.359 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406e960 msgr2=0x7f28041553e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406e960 0x7f28041553e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f27f400c5b0 tx=0x7f27f4001070 comp rx=0 tx=0).stop 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 shutdown_connections 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f280406e960 0x7f28041553e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f28041560c0 0x7f28041679b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 --2- 172.21.15.123:0/1864923372 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f280406f080 0x7f280415a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 >> 172.21.15.123:0/1864923372 conn(0x7f28040cda80 msgr2=0x7f280406a370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 shutdown_connections 2023-10-06T04:31:13.360 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:13.357+0000 7f280b4f0700 1 -- 172.21.15.123:0/1864923372 wait complete. 2023-10-06T04:31:13.370 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T04:31:13.670 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:13.667+0000 7fab376311c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:31:13.781 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:13.779+0000 7fd68d5c31c0 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-10-06T04:31:13.980 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:13.978+0000 7fab376311c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:31:14.099 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:14.096+0000 7fd68d5c31c0 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-10-06T04:31:14.119 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:14.116+0000 7fab376311c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:31:14.238 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:14.236+0000 7fd68d5c31c0 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:31:14.472 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:14.470+0000 7fab376311c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:31:14.592 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:14.590+0000 7fd68d5c31c0 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-10-06T04:31:14.613 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:14.611+0000 7fab376311c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:31:14.736 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:14.734+0000 7fd68d5c31c0 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-10-06T04:31:14.762 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:14.760+0000 7fab376311c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:31:14.861 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:14.859+0000 7fab376311c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:31:14.884 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:14.882+0000 7fd68d5c31c0 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-10-06T04:31:14.988 INFO:tasks.ceph.mgr.x.smithi177.stderr:2023-10-06T04:31:14.986+0000 7fd68d5c31c0 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-10-06T04:31:15.884 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:15.882+0000 7faafbd12700 -1 mgr.server handle_report got status from non-daemon mon.b 2023-10-06T04:31:15.885 INFO:tasks.ceph.mgr.y.smithi123.stderr:2023-10-06T04:31:15.883+0000 7faafbd12700 -1 mgr.server handle_report got status from non-daemon mon.c 2023-10-06T04:31:16.200 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T04:31:16.198+0000 7fe6c2ace540 -1 osd.5 0 log_to_monitors true 2023-10-06T04:31:16.224 INFO:tasks.ceph.osd.3.smithi123.stderr:2023-10-06T04:31:16.222+0000 7f62169d3540 -1 osd.3 0 log_to_monitors true 2023-10-06T04:31:16.229 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T04:31:16.227+0000 7ff5fbc80540 -1 osd.7 0 log_to_monitors true 2023-10-06T04:31:16.246 INFO:tasks.ceph.osd.0.smithi123.stderr:2023-10-06T04:31:16.243+0000 7f640f304540 -1 osd.0 0 log_to_monitors true 2023-10-06T04:31:16.254 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T04:31:16.252+0000 7f7706942540 -1 osd.4 0 log_to_monitors true 2023-10-06T04:31:16.254 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T04:31:16.252+0000 7f37f888c540 -1 osd.6 0 log_to_monitors true 2023-10-06T04:31:16.261 INFO:tasks.ceph.osd.2.smithi123.stderr:2023-10-06T04:31:16.259+0000 7ff067196540 -1 osd.2 0 log_to_monitors true 2023-10-06T04:31:16.271 INFO:tasks.ceph.osd.1.smithi123.stderr:2023-10-06T04:31:16.269+0000 7fd099854540 -1 osd.1 0 log_to_monitors true 2023-10-06T04:31:19.853 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T04:31:19.851+0000 7f76fd547700 -1 osd.4 0 waiting for initial osdmap 2023-10-06T04:31:19.861 INFO:tasks.ceph.osd.4.smithi177.stderr:2023-10-06T04:31:19.859+0000 7f76f633e700 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:19.865 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T04:31:19.863+0000 7fe6b96d3700 -1 osd.5 0 waiting for initial osdmap 2023-10-06T04:31:19.877 INFO:tasks.ceph.osd.5.smithi177.stderr:2023-10-06T04:31:19.875+0000 7fe6b24ca700 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:19.897 INFO:tasks.ceph.osd.3.smithi123.stderr:2023-10-06T04:31:19.895+0000 7f620eddb700 -1 osd.3 0 waiting for initial osdmap 2023-10-06T04:31:19.902 INFO:tasks.ceph.osd.3.smithi123.stderr:2023-10-06T04:31:19.900+0000 7f62063cf700 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:19.915 INFO:tasks.ceph.osd.2.smithi123.stderr:2023-10-06T04:31:19.913+0000 7ff05dd9b700 -1 osd.2 0 waiting for initial osdmap 2023-10-06T04:31:19.915 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T04:31:19.914+0000 7ff5f2885700 -1 osd.7 0 waiting for initial osdmap 2023-10-06T04:31:19.919 INFO:tasks.ceph.osd.2.smithi123.stderr:2023-10-06T04:31:19.917+0000 7ff056b92700 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:19.921 INFO:tasks.ceph.osd.7.smithi177.stderr:2023-10-06T04:31:19.919+0000 7ff5eb67c700 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:19.922 INFO:tasks.ceph.osd.1.smithi123.stderr:2023-10-06T04:31:19.920+0000 7fd090459700 -1 osd.1 0 waiting for initial osdmap 2023-10-06T04:31:19.929 INFO:tasks.ceph.osd.1.smithi123.stderr:2023-10-06T04:31:19.927+0000 7fd089250700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:19.943 INFO:tasks.ceph.osd.0.smithi123.stderr:2023-10-06T04:31:19.941+0000 7f6405f09700 -1 osd.0 0 waiting for initial osdmap 2023-10-06T04:31:19.946 INFO:tasks.ceph.osd.0.smithi123.stderr:2023-10-06T04:31:19.944+0000 7f63fed00700 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:20.176 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:20.236 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T04:31:20.234+0000 7f37ef491700 -1 osd.6 0 waiting for initial osdmap 2023-10-06T04:31:20.239 INFO:tasks.ceph.osd.6.smithi177.stderr:2023-10-06T04:31:20.237+0000 7f37e8288700 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-10-06T04:31:20.286 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.283+0000 7fc49dfc5700 1 Processor -- start 2023-10-06T04:31:20.286 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.283+0000 7fc49dfc5700 1 -- start start 2023-10-06T04:31:20.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc49dfc5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc4980688c0 0x7fc4980c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:20.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc49dfc5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc498068c90 0x7fc49806e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:20.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc49dfc5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc498069a30 con 0x7fc49806e970 2023-10-06T04:31:20.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc49dfc5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc49806a890 con 0x7fc498068c90 2023-10-06T04:31:20.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc49dfc5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc498069c80 con 0x7fc4980688c0 2023-10-06T04:31:20.288 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc496ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc498068c90 0x7fc49806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc496ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc498068c90 0x7fc49806e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45522/0 (socket says 172.21.15.123:45522) 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc496ffd700 1 -- 172.21.15.123:0/2500786323 learned_addr learned my addr 172.21.15.123:0/2500786323 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.284+0000 7fc4977fe700 1 --2- 172.21.15.123:0/2500786323 >> v2:172.21.15.177:3300/0 conn(0x7fc4980688c0 0x7fc4980c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc496ffd700 1 -- 172.21.15.123:0/2500786323 >> v2:172.21.15.177:3300/0 conn(0x7fc4980688c0 msgr2=0x7fc4980c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc496ffd700 1 --2- 172.21.15.123:0/2500786323 >> v2:172.21.15.177:3300/0 conn(0x7fc4980688c0 0x7fc4980c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc496ffd700 1 -- 172.21.15.123:0/2500786323 >> v1:172.21.15.177:6789/0 conn(0x7fc49806e970 legacy=0x7fc498158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc496ffd700 1 -- 172.21.15.123:0/2500786323 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc498069f60 con 0x7fc498068c90 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc496ffd700 1 --2- 172.21.15.123:0/2500786323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc49806e430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc49806f2f0 tx=0x7fc488009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15a56b35e7d5b70 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc4967fc700 1 -- 172.21.15.123:0/2500786323 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc488009e70 con 0x7fc498068c90 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc4967fc700 1 -- 172.21.15.123:0/2500786323 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc48800b860 con 0x7fc498068c90 2023-10-06T04:31:20.289 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc4967fc700 1 -- 172.21.15.123:0/2500786323 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800bc00 con 0x7fc498068c90 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc49dfc5700 1 -- 172.21.15.123:0/2500786323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 msgr2=0x7fc49806e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/2500786323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc49806e430 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc49806f2f0 tx=0x7fc488009250 comp rx=0 tx=0).stop 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc49dfc5700 1 -- 172.21.15.123:0/2500786323 shutdown_connections 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/2500786323 >> v2:172.21.15.177:3300/0 conn(0x7fc4980688c0 0x7fc4980c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/2500786323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc49806e430 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc49dfc5700 1 -- 172.21.15.123:0/2500786323 >> 172.21.15.123:0/2500786323 conn(0x7fc4980c9c90 msgr2=0x7fc49806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.285+0000 7fc49dfc5700 1 -- 172.21.15.123:0/2500786323 shutdown_connections 2023-10-06T04:31:20.290 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.286+0000 7fc49dfc5700 1 -- 172.21.15.123:0/2500786323 wait complete. 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.287+0000 7fc49dfc5700 1 Processor -- start 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.287+0000 7fc49dfc5700 1 -- start start 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.287+0000 7fc49dfc5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4980688c0 0x7fc4981481e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.287+0000 7fc49dfc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc498158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.287+0000 7fc49dfc5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc49806e970 0x7fc4980c2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc49dfc5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc498159800 con 0x7fc498068c90 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc49dfc5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc498159ae0 con 0x7fc49806e970 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc49dfc5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc498159dc0 con 0x7fc4980688c0 2023-10-06T04:31:20.293 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc498158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc498158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45538/0 (socket says 172.21.15.123:45538) 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 -- 172.21.15.123:0/1664931654 learned_addr learned my addr 172.21.15.123:0/1664931654 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc4977fe700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4980688c0 0x7fc4981481e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc497fff700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc49806e970 0x7fc4980c2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 -- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4980688c0 msgr2=0x7fc4981481e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4980688c0 0x7fc4981481e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 -- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc49806e970 msgr2=0x7fc4980c2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc49806e970 0x7fc4980c2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 -- 172.21.15.123:0/1664931654 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4880094f0 con 0x7fc498068c90 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc496ffd700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc498158d40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc49806d230 tx=0x7fc488017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc47f7fe700 1 -- 172.21.15.123:0/1664931654 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc48800bbe0 con 0x7fc498068c90 2023-10-06T04:31:20.294 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc47f7fe700 1 -- 172.21.15.123:0/1664931654 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc488011420 con 0x7fc498068c90 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc49815a0a0 con 0x7fc498068c90 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc47f7fe700 1 -- 172.21.15.123:0/1664931654 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4880117c0 con 0x7fc498068c90 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.288+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc498148720 con 0x7fc498068c90 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.289+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc498070f10 con 0x7fc498068c90 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.289+0000 7fc47f7fe700 1 -- 172.21.15.123:0/1664931654 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 4) v1 ==== 84684+0+0 (secure 0 0 0) 0x7fc488011bf0 con 0x7fc498068c90 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.290+0000 7fc47f7fe700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc484066ed0 0x7fc484069380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.290+0000 7fc4977fe700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc484066ed0 0x7fc484069380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.290+0000 7fc47f7fe700 1 -- 172.21.15.123:0/1664931654 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2227+0+0 (secure 0 0 0) 0x7fc488084af0 con 0x7fc498068c90 2023-10-06T04:31:20.295 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.290+0000 7fc4977fe700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc484066ed0 0x7fc484069380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009db0 tx=0x7fc48c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:20.297 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.295+0000 7fc47f7fe700 1 -- 172.21.15.123:0/1664931654 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc498070f10 con 0x7fc498068c90 2023-10-06T04:31:20.480 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.478+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc498070f10 con 0x7fc498068c90 2023-10-06T04:31:20.481 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.479+0000 7fc47f7fe700 1 -- 172.21.15.123:0/1664931654 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7013 (secure 0 0 0) 0x7fc498070f10 con 0x7fc498068c90 2023-10-06T04:31:20.481 INFO:teuthology.misc.health.smithi123.stdout: 2023-10-06T04:31:20.481 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":12,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:17.885727+0000","last_up_change":"0.000000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:20.483 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.480+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc484066ed0 msgr2=0x7fc484069380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:20.483 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.480+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc484066ed0 0x7fc484069380 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009db0 tx=0x7fc48c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.483 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.480+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 msgr2=0x7fc498158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:20.483 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.480+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc498158d40 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fc49806d230 tx=0x7fc488017000 comp rx=0 tx=0).stop 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 shutdown_connections 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc484066ed0 0x7fc484069380 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc49806e970 0x7fc4980c2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc4980688c0 0x7fc4981481e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 --2- 172.21.15.123:0/1664931654 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc498068c90 0x7fc498158d40 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 >> 172.21.15.123:0/1664931654 conn(0x7fc4980c9c90 msgr2=0x7fc4980af6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 shutdown_connections 2023-10-06T04:31:20.484 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:20.481+0000 7fc49dfc5700 1 -- 172.21.15.123:0/1664931654 wait complete. 2023-10-06T04:31:20.494 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-10-06T04:31:27.300 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:27.421 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.419+0000 7f624bb52700 1 Processor -- start 2023-10-06T04:31:27.421 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.420+0000 7f624bb52700 1 -- start start 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624bb52700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f62441574f0 0x7f6244161fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624bb52700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62440cc190 con 0x7f62440688c0 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624bb52700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f624406ab00 con 0x7f6244068c90 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624bb52700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6244157950 con 0x7f62441574f0 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624a0ef700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f62441574f0 0x7f6244161fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f62498ee700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f62440688c0 0x7f624406a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:59658/0 (socket says 172.21.15.123:59658) 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f62498ee700 1 -- 172.21.15.123:0/3855302165 learned_addr learned my addr 172.21.15.123:0/3855302165 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624a0ef700 1 -- 172.21.15.123:0/3855302165 >> v1:172.21.15.177:6789/0 conn(0x7f6244068c90 legacy=0x7f6244156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624a0ef700 1 -- 172.21.15.123:0/3855302165 >> v1:172.21.15.123:6789/0 conn(0x7f62440688c0 legacy=0x7f624406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.423 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f62488ec700 1 -- 172.21.15.123:0/3855302165 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1303674712 0 0) 0x7f62440cc190 con 0x7f62440688c0 2023-10-06T04:31:27.424 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624a0ef700 1 -- 172.21.15.123:0/3855302165 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62440cb580 con 0x7f62441574f0 2023-10-06T04:31:27.424 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f624a0ef700 1 --2- 172.21.15.123:0/3855302165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62441574f0 0x7f6244161fe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f623c009e60 tx=0x7f623c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e433bb3e6fe27ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:27.424 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f62488ec700 1 -- 172.21.15.123:0/3855302165 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c00e070 con 0x7f62441574f0 2023-10-06T04:31:27.424 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f62488ec700 1 -- 172.21.15.123:0/3855302165 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f623c00bbf0 con 0x7f62441574f0 2023-10-06T04:31:27.424 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.421+0000 7f62488ec700 1 -- 172.21.15.123:0/3855302165 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f623c012430 con 0x7f62441574f0 2023-10-06T04:31:27.426 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.422+0000 7f624bb52700 1 -- 172.21.15.123:0/3855302165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62441574f0 msgr2=0x7f6244161fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.426 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.422+0000 7f624bb52700 1 --2- 172.21.15.123:0/3855302165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62441574f0 0x7f6244161fe0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f623c009e60 tx=0x7f623c009780 comp rx=0 tx=0).stop 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.422+0000 7f624bb52700 1 -- 172.21.15.123:0/3855302165 shutdown_connections 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.422+0000 7f624bb52700 1 --2- 172.21.15.123:0/3855302165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f62441574f0 0x7f6244161fe0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.422+0000 7f624bb52700 1 -- 172.21.15.123:0/3855302165 >> 172.21.15.123:0/3855302165 conn(0x7f62440c9c90 msgr2=0x7f624406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.422+0000 7f624bb52700 1 -- 172.21.15.123:0/3855302165 shutdown_connections 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.422+0000 7f624bb52700 1 -- 172.21.15.123:0/3855302165 wait complete. 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.423+0000 7f624bb52700 1 Processor -- start 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.423+0000 7f624bb52700 1 -- start start 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624bb52700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62440688c0 0x7f6244157a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624bb52700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6244068c90 0x7f6244169e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624bb52700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62441574f0 0x7f6244159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.427 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624bb52700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62440ae210 con 0x7f6244068c90 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624bb52700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f624416a350 con 0x7f62440688c0 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624bb52700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6244158d80 con 0x7f62441574f0 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6244068c90 0x7f6244169e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6244068c90 0x7f6244169e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45598/0 (socket says 172.21.15.123:45598) 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 -- 172.21.15.123:0/3219489395 learned_addr learned my addr 172.21.15.123:0/3219489395 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624a0ef700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62441574f0 0x7f6244159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62498ee700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62440688c0 0x7f6244157a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 -- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62441574f0 msgr2=0x7f6244159210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62441574f0 0x7f6244159210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 -- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62440688c0 msgr2=0x7f6244157a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62440688c0 0x7f6244157a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.428 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62490ed700 1 -- 172.21.15.123:0/3219489395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f623c009a20 con 0x7f6244068c90 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f62498ee700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62440688c0 0x7f6244157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.424+0000 7f624a0ef700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62441574f0 0x7f6244159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.425+0000 7f62490ed700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6244068c90 0x7f6244169e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f624000c5d0 tx=0x7f6240007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.425+0000 7f6239ffb700 1 -- 172.21.15.123:0/3219489395 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240011590 con 0x7f6244068c90 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.425+0000 7f6239ffb700 1 -- 172.21.15.123:0/3219489395 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6240011710 con 0x7f6244068c90 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.425+0000 7f6239ffb700 1 -- 172.21.15.123:0/3219489395 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6240011ad0 con 0x7f6244068c90 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.425+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6244158170 con 0x7f6244068c90 2023-10-06T04:31:27.429 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.425+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6244158730 con 0x7f6244068c90 2023-10-06T04:31:27.430 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.426+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f624406eb30 con 0x7f6244068c90 2023-10-06T04:31:27.430 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.426+0000 7f6239ffb700 1 -- 172.21.15.123:0/3219489395 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f624001c460 con 0x7f6244068c90 2023-10-06T04:31:27.430 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.426+0000 7f6239ffb700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6228066f30 0x7f62280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.430 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.427+0000 7f62498ee700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6228066f30 0x7f62280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.430 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.427+0000 7f6239ffb700 1 -- 172.21.15.123:0/3219489395 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f6240083f40 con 0x7f6244068c90 2023-10-06T04:31:27.430 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.427+0000 7f62498ee700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6228066f30 0x7f62280693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62340011e0 tx=0x7f6234009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:27.432 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.431+0000 7f6239ffb700 1 -- 172.21.15.123:0/3219489395 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6240051500 con 0x7f6244068c90 2023-10-06T04:31:27.615 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.613+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6244158ef0 con 0x7f6244068c90 2023-10-06T04:31:27.615 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.614+0000 7f6239ffb700 1 -- 172.21.15.123:0/3219489395 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v16) v1 ==== 74+0+12271 (secure 0 0 0) 0x7f6240054b20 con 0x7f6244068c90 2023-10-06T04:31:27.615 INFO:teuthology.misc.health.smithi123.stdout: 2023-10-06T04:31:27.616 INFO:teuthology.misc.health.smithi123.stdout:{"epoch":16,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:23.879196+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:27.617 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6228066f30 msgr2=0x7f62280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6228066f30 0x7f62280693e0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f62340011e0 tx=0x7f6234009040 comp rx=0 tx=0).stop 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6244068c90 msgr2=0x7f6244169e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6244068c90 0x7f6244169e10 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f624000c5d0 tx=0x7f6240007f60 comp rx=0 tx=0).stop 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 shutdown_connections 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6228066f30 0x7f62280693e0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62441574f0 0x7f6244159210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62440688c0 0x7f6244157a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 --2- 172.21.15.123:0/3219489395 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6244068c90 0x7f6244169e10 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 >> 172.21.15.123:0/3219489395 conn(0x7f62440c9c90 msgr2=0x7f6244069a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:27.618 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 shutdown_connections 2023-10-06T04:31:27.619 INFO:teuthology.misc.health.smithi123.stderr:2023-10-06T04:31:27.616+0000 7f624bb52700 1 -- 172.21.15.123:0/3219489395 wait complete. 2023-10-06T04:31:27.629 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-10-06T04:31:27.629 INFO:tasks.ceph:Creating RBD pool 2023-10-06T04:31:27.629 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd pool create rbd 8 2023-10-06T04:31:27.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.745+0000 7f0211b48700 1 Processor -- start 2023-10-06T04:31:27.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.745+0000 7f0211b48700 1 -- start start 2023-10-06T04:31:27.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f0211b48700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 0x7f020c0a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f0211b48700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c069aa0 con 0x7f020c068d80 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f0211b48700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020c06a340 con 0x7f020c0c8710 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f0211b48700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0aa6f0 con 0x7f020c0c8340 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f020c068d80 0x7f020c06a7d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:59672/0 (socket says 172.21.15.123:59672) 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020bfff700 1 -- 172.21.15.123:0/3815178567 learned_addr learned my addr 172.21.15.123:0/3815178567 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020b7fe700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 0x7f020c0a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3114774616 0 0) 0x7f020c069aa0 con 0x7f020c068d80 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01f4003570 con 0x7f020c068d80 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 222750276 0 0) 0x7f01f4003570 con 0x7f020c068d80 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 msgr2=0x7f020c0a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:27.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 0x7f020c0a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 >> v1:172.21.15.177:6789/0 conn(0x7f020c0c8710 legacy=0x7f020c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c069cf0 con 0x7f020c068d80 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.746+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f02040017f0 con 0x7f020c068d80 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 >> v1:172.21.15.123:6789/0 conn(0x7f020c068d80 legacy=0x7f020c06a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4003a90 0x7f01f4003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4004810 0x7f01f4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 0x7f01f40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0aa6f0 con 0x7f01f4004810 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01f400bb10 con 0x7f01f4003a90 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01f400bcd0 con 0x7f020c0c8340 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020affd700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4004810 0x7f01f4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020b7fe700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4003a90 0x7f01f4003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020bfff700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 0x7f01f40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020affd700 1 -- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 msgr2=0x7f01f40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020affd700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c0c8340 0x7f01f40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020affd700 1 -- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4003a90 msgr2=0x7f01f4003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020affd700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4003a90 0x7f01f4003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020affd700 1 -- 172.21.15.123:0/3815178567 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01f4003570 con 0x7f01f4004810 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020affd700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4004810 0x7f01f4004c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0019a0 tx=0x7f01fc007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7211a3dc001c6e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00c5d0 con 0x7f01f4004810 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc00c750 con 0x7f01f4004810 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.747+0000 7f020a7fc700 1 -- 172.21.15.123:0/3815178567 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc00cb10 con 0x7f01f4004810 2023-10-06T04:31:27.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.748+0000 7f0211b48700 1 -- 172.21.15.123:0/3815178567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4004810 msgr2=0x7f01f4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.748+0000 7f0211b48700 1 --2- 172.21.15.123:0/3815178567 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01f4004810 0x7f01f4004c80 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f01fc0019a0 tx=0x7f01fc007d10 comp rx=0 tx=0).stop 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.748+0000 7f020bfff700 1 -- 172.21.15.123:0/3815178567 reap_dead start 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.748+0000 7f0211b48700 1 -- 172.21.15.123:0/3815178567 shutdown_connections 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.748+0000 7f0211b48700 1 -- 172.21.15.123:0/3815178567 >> 172.21.15.123:0/3815178567 conn(0x7f020c0c9ca0 msgr2=0x7f020c06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.748+0000 7f0211b48700 1 -- 172.21.15.123:0/3815178567 shutdown_connections 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.748+0000 7f0211b48700 1 -- 172.21.15.123:0/3815178567 wait complete. 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.749+0000 7f0211b48700 1 Processor -- start 2023-10-06T04:31:27.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.749+0000 7f0211b48700 1 -- start start 2023-10-06T04:31:27.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4004810 0x7f020c152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c1565d0 0x7f020c0b0600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f020c0b0b40 0x7f020c0abe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0b0fb0 con 0x7f020c0b0b40 2023-10-06T04:31:27.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0ad500 con 0x7f01f4004810 2023-10-06T04:31:27.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f020c0ad7e0 con 0x7f020c1565d0 2023-10-06T04:31:27.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f020c0b0b40 0x7f020c0abe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f020c0b0b40 0x7f020c0abe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45616/0 (socket says 172.21.15.123:45616) 2023-10-06T04:31:27.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 -- 172.21.15.123:0/2801988528 learned_addr learned my addr 172.21.15.123:0/2801988528 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:27.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020b7fe700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4004810 0x7f020c152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020affd700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c1565d0 0x7f020c0b0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 -- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c1565d0 msgr2=0x7f020c0b0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c1565d0 0x7f020c0b0600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 -- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4004810 msgr2=0x7f020c152030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4004810 0x7f020c152030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 -- 172.21.15.123:0/2801988528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020c0adac0 con 0x7f020c0b0b40 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f020bfff700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f020c0b0b40 0x7f020c0abe90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0204000f80 tx=0x7f020c0ab640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f01f37fe700 1 -- 172.21.15.123:0/2801988528 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f020400ad80 con 0x7f020c0b0b40 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f01f37fe700 1 -- 172.21.15.123:0/2801988528 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02040133f0 con 0x7f020c0b0b40 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f01f37fe700 1 -- 172.21.15.123:0/2801988528 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02040137b0 con 0x7f020c0b0b40 2023-10-06T04:31:27.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f020c0adda0 con 0x7f020c0b0b40 2023-10-06T04:31:27.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.750+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f020c156a40 con 0x7f020c0b0b40 2023-10-06T04:31:27.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.751+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f020c0c8be0 con 0x7f020c0b0b40 2023-10-06T04:31:27.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.752+0000 7f01f37fe700 1 -- 172.21.15.123:0/2801988528 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0204013b60 con 0x7f020c0b0b40 2023-10-06T04:31:27.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.752+0000 7f01f37fe700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01f8066f30 0x7f01f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:27.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.752+0000 7f01f37fe700 1 -- 172.21.15.123:0/2801988528 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(16..16 src has 1..16) v4 ==== 4853+0+0 (secure 0 0 0) 0x7f0204084a50 con 0x7f020c0b0b40 2023-10-06T04:31:27.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.752+0000 7f020b7fe700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01f8066f30 0x7f01f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:27.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.752+0000 7f020b7fe700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01f8066f30 0x7f01f80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0200006f00 tx=0x7f02000091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:27.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.756+0000 7f01f37fe700 1 -- 172.21.15.123:0/2801988528 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0204052010 con 0x7f020c0b0b40 2023-10-06T04:31:27.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:27.937+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "rbd", "pg_num": 8} v 0) v1 -- 0x7f020c0ca0a0 con 0x7f020c0b0b40 2023-10-06T04:31:28.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.010+0000 7f01f37fe700 1 -- 172.21.15.123:0/2801988528 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "rbd", "pg_num": 8}]=0 pool 'rbd' created v17) v1 ==== 109+0+0 (secure 0 0 0) 0x7f0204055630 con 0x7f020c0b0b40 2023-10-06T04:31:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01f8066f30 msgr2=0x7f01f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01f8066f30 0x7f01f80693e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0200006f00 tx=0x7f02000091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f020c0b0b40 msgr2=0x7f020c0abe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f020c0b0b40 0x7f020c0abe90 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0204000f80 tx=0x7f020c0ab640 comp rx=0 tx=0).stop 2023-10-06T04:31:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 shutdown_connections 2023-10-06T04:31:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01f8066f30 0x7f01f80693e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f020c0b0b40 0x7f020c0abe90 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01f4004810 0x7f020c152030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 --2- 172.21.15.123:0/2801988528 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f020c1565d0 0x7f020c0b0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 >> 172.21.15.123:0/2801988528 conn(0x7f020c0c9ca0 msgr2=0x7f020c0cb2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:28.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 shutdown_connections 2023-10-06T04:31:28.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.012+0000 7f0211b48700 1 -- 172.21.15.123:0/2801988528 wait complete. 2023-10-06T04:31:28.016 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'rbd' created 2023-10-06T04:31:28.026 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2023-10-06T04:31:28.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.144+0000 7f0ba5d67700 1 Processor -- start 2023-10-06T04:31:28.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.144+0000 7f0ba5d67700 1 -- start start 2023-10-06T04:31:28.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0ba5d67700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00cb650 0x7f0ba015a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:28.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0ba5d67700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ba00aec90 0x7f0ba0067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:28.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0ba5d67700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ba00634b0 con 0x7f0ba00cba20 2023-10-06T04:31:28.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0ba5d67700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ba00c9aa0 con 0x7f0ba00aec90 2023-10-06T04:31:28.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0ba5d67700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c9d80 con 0x7f0ba00cb650 2023-10-06T04:31:28.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0b9f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00cb650 0x7f0ba015a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0b9effd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ba00aec90 0x7f0ba0067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0b9ffff700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0ba00cba20 0x7f0ba0155b10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:46652/0 (socket says 172.21.15.123:46652) 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0b9ffff700 1 -- 172.21.15.123:0/3011335046 learned_addr learned my addr 172.21.15.123:0/3011335046 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0b9f7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00cb650 0x7f0ba015a680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:42174/0 (socket says 172.21.15.123:42174) 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.145+0000 7f0b9effd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0ba00aec90 0x7f0ba0067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:58482/0 (socket says 172.21.15.123:58482) 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9e7fc700 1 -- 172.21.15.123:0/3011335046 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782061445 0 0) 0x7f0ba00634b0 con 0x7f0ba00cba20 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9e7fc700 1 -- 172.21.15.123:0/3011335046 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0b88003570 con 0x7f0ba00cba20 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9effd700 1 -- 172.21.15.123:0/3011335046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00cb650 msgr2=0x7f0ba015a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9effd700 1 --2- 172.21.15.123:0/3011335046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00cb650 0x7f0ba015a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9effd700 1 -- 172.21.15.123:0/3011335046 >> v1:172.21.15.177:6789/0 conn(0x7f0ba00cba20 legacy=0x7f0ba0155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:28.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9effd700 1 -- 172.21.15.123:0/3011335046 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ba00c7cc0 con 0x7f0ba00aec90 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9effd700 1 --2- 172.21.15.123:0/3011335046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00aec90 0x7f0ba0067660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b940098d0 tx=0x7f0b94009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b2766f3bb12a5df2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9e7fc700 1 -- 172.21.15.123:0/3011335046 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94005c10 con 0x7f0ba00aec90 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9e7fc700 1 -- 172.21.15.123:0/3011335046 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b94005d70 con 0x7f0ba00aec90 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.146+0000 7f0b9e7fc700 1 -- 172.21.15.123:0/3011335046 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b94006260 con 0x7f0ba00aec90 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3011335046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00aec90 msgr2=0x7f0ba0067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3011335046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00aec90 0x7f0ba0067660 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0b940098d0 tx=0x7f0b94009120 comp rx=0 tx=0).stop 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3011335046 shutdown_connections 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3011335046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00cb650 0x7f0ba015a680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3011335046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00aec90 0x7f0ba0067660 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3011335046 >> 172.21.15.123:0/3011335046 conn(0x7f0ba00cd250 msgr2=0x7f0ba00c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:28.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3011335046 shutdown_connections 2023-10-06T04:31:28.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.147+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3011335046 wait complete. 2023-10-06T04:31:28.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.148+0000 7f0ba5d67700 1 Processor -- start 2023-10-06T04:31:28.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.148+0000 7f0ba5d67700 1 -- start start 2023-10-06T04:31:28.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0ba5d67700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00aec90 0x7f0ba0169c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:28.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0ba5d67700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00cb650 0x7f0ba00c5f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:28.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0ba5d67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba00cba20 0x7f0ba0069a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0ba5d67700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba00c3830 con 0x7f0ba00cba20 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0ba5d67700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba016a520 con 0x7f0ba00cb650 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0ba5d67700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ba016a800 con 0x7f0ba00aec90 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba00cba20 0x7f0ba0069a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba00cba20 0x7f0ba0069a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45618/0 (socket says 172.21.15.123:45618) 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 -- 172.21.15.123:0/3013368903 learned_addr learned my addr 172.21.15.123:0/3013368903 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9f7fe700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00aec90 0x7f0ba0169c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9effd700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00cb650 0x7f0ba00c5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 -- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00aec90 msgr2=0x7f0ba0169c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00aec90 0x7f0ba0169c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 -- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00cb650 msgr2=0x7f0ba00c5f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00cb650 0x7f0ba00c5f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 -- 172.21.15.123:0/3013368903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b940093c0 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.149+0000 7f0b9ffff700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba00cba20 0x7f0ba0069a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001460 tx=0x7f0b98001ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.150+0000 7f0b877fe700 1 -- 172.21.15.123:0/3013368903 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800a890 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.150+0000 7f0b877fe700 1 -- 172.21.15.123:0/3013368903 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b9800a9f0 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.150+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ba016aae0 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.150+0000 7f0b877fe700 1 -- 172.21.15.123:0/3013368903 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b9800ad70 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.150+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ba016b170 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.151+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ba016b300 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.151+0000 7f0b877fe700 1 -- 172.21.15.123:0/3013368903 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b98007300 con 0x7f0ba00cba20 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.151+0000 7f0b877fe700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b8c066f30 0x7f0b8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:28.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.152+0000 7f0b877fe700 1 -- 172.21.15.123:0/3013368903 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 5195+0+0 (secure 0 0 0) 0x7f0b98084700 con 0x7f0ba00cba20 2023-10-06T04:31:28.156 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.152+0000 7f0b9f7fe700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b8c066f30 0x7f0b8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:28.156 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.152+0000 7f0b9f7fe700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b8c066f30 0x7f0b8c0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009d10 tx=0x7f0b900091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.156+0000 7f0b877fe700 1 -- 172.21.15.123:0/3013368903 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b98051b60 con 0x7f0ba00cba20 2023-10-06T04:31:28.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:28.339+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f0ba00c7fa0 con 0x7f0ba00cba20 2023-10-06T04:31:29.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.014+0000 7f0b877fe700 1 -- 172.21.15.123:0/3013368903 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool application enable", "pool": "rbd", "app": "rbd", "yes_i_really_mean_it": true}]=0 enabled application 'rbd' on pool 'rbd' v18) v1 ==== 173+0+0 (secure 0 0 0) 0x7f0b98055180 con 0x7f0ba00cba20 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b8c066f30 msgr2=0x7f0b8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b8c066f30 0x7f0b8c0693e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0b90009d10 tx=0x7f0b900091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba00cba20 msgr2=0x7f0ba0069a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba00cba20 0x7f0ba0069a40 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f0b98001460 tx=0x7f0b98001ae0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 shutdown_connections 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b8c066f30 0x7f0b8c0693e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0ba00cba20 0x7f0ba0069a40 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0ba00aec90 0x7f0ba0169c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 --2- 172.21.15.123:0/3013368903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0ba00cb650 0x7f0ba00c5f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 >> 172.21.15.123:0/3013368903 conn(0x7f0ba00cd250 msgr2=0x7f0ba014d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.017+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 shutdown_connections 2023-10-06T04:31:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.018+0000 7f0ba5d67700 1 -- 172.21.15.123:0/3013368903 wait complete. 2023-10-06T04:31:29.020 INFO:teuthology.orchestra.run.smithi123.stderr:enabled application 'rbd' on pool 'rbd' 2023-10-06T04:31:29.034 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-10-06T04:31:29.034 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:31:29.034 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:31:29.036 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:31:29.036 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:31:29.037 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:31:29.038 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:31:29.038 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:31:29.039 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:31:29.041 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:31:29.041 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:31:29.041 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:31:29.043 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:31:29.043 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-10-06T04:31:29.044 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:29.121 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2023-10-06T04:31:29.121 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2023-10-06T04:31:29.125 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-10-06T04:31:29.126 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-10-06T04:31:29.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.173+0000 7fe527f16700 1 Processor -- start 2023-10-06T04:31:29.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.173+0000 7fe527f16700 1 -- start start 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe527f16700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006a540 0x7fe52015d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe527f16700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5200690a0 con 0x7fe52006bb60 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe527f16700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe52015dd40 con 0x7fe52013f600 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe527f16700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe520068770 con 0x7fe52006a540 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe5254b1700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe52006bb60 0x7fe52015d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:59694/0 (socket says 172.21.15.123:59694) 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe5254b1700 1 -- 172.21.15.123:0/1740827482 learned_addr learned my addr 172.21.15.123:0/1740827482 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe5264b3700 1 --2- 172.21.15.123:0/1740827482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006a540 0x7fe52015d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe524cb0700 1 -- 172.21.15.123:0/1740827482 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3012334800 0 0) 0x7fe5200690a0 con 0x7fe52006bb60 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe524cb0700 1 -- 172.21.15.123:0/1740827482 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe50c003570 con 0x7fe52006bb60 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe524cb0700 1 -- 172.21.15.123:0/1740827482 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666798277 0 0) 0x7fe52015dd40 con 0x7fe52013f600 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe524cb0700 1 -- 172.21.15.123:0/1740827482 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe5200690a0 con 0x7fe52013f600 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe5264b3700 1 -- 172.21.15.123:0/1740827482 >> v1:172.21.15.177:6789/0 conn(0x7fe52013f600 legacy=0x7fe52006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe5264b3700 1 -- 172.21.15.123:0/1740827482 >> v1:172.21.15.123:6789/0 conn(0x7fe52006bb60 legacy=0x7fe52015d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.174+0000 7fe5264b3700 1 -- 172.21.15.123:0/1740827482 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe520068a50 con 0x7fe52006a540 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe5264b3700 1 --2- 172.21.15.123:0/1740827482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006a540 0x7fe52015d720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe518009e90 tx=0x7fe518009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a13e61de081f96c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe524cb0700 1 -- 172.21.15.123:0/1740827482 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51800e070 con 0x7fe52006a540 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe524cb0700 1 -- 172.21.15.123:0/1740827482 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51800bb80 con 0x7fe52006a540 2023-10-06T04:31:29.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe524cb0700 1 -- 172.21.15.123:0/1740827482 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe518012430 con 0x7fe52006a540 2023-10-06T04:31:29.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe527f16700 1 -- 172.21.15.123:0/1740827482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006a540 msgr2=0x7fe52015d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe527f16700 1 --2- 172.21.15.123:0/1740827482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006a540 0x7fe52015d720 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fe518009e90 tx=0x7fe518009780 comp rx=0 tx=0).stop 2023-10-06T04:31:29.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe527f16700 1 -- 172.21.15.123:0/1740827482 shutdown_connections 2023-10-06T04:31:29.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe527f16700 1 --2- 172.21.15.123:0/1740827482 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006a540 0x7fe52015d720 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe527f16700 1 -- 172.21.15.123:0/1740827482 >> 172.21.15.123:0/1740827482 conn(0x7fe5200ce0c0 msgr2=0x7fe5200ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe527f16700 1 -- 172.21.15.123:0/1740827482 shutdown_connections 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.175+0000 7fe527f16700 1 -- 172.21.15.123:0/1740827482 wait complete. 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.176+0000 7fe527f16700 1 Processor -- start 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.176+0000 7fe527f16700 1 -- start start 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe527f16700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006bb60 0x7fe520155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe527f16700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe52013f600 0x7fe52015c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe527f16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52016a5d0 0x7fe520169880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe527f16700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe52013dd50 con 0x7fe52016a5d0 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe527f16700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201599a0 con 0x7fe52013f600 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe527f16700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5201564f0 con 0x7fe52006bb60 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe5264b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52016a5d0 0x7fe520169880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe5264b3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52016a5d0 0x7fe520169880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45666/0 (socket says 172.21.15.123:45666) 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe5264b3700 1 -- 172.21.15.123:0/2433831598 learned_addr learned my addr 172.21.15.123:0/2433831598 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe5254b1700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe52013f600 0x7fe52015c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.177+0000 7fe525cb2700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006bb60 0x7fe520155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe5264b3700 1 -- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006bb60 msgr2=0x7fe520155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe5264b3700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006bb60 0x7fe520155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe5264b3700 1 -- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe52013f600 msgr2=0x7fe52015c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe5264b3700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe52013f600 0x7fe52015c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe5264b3700 1 -- 172.21.15.123:0/2433831598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe518009a20 con 0x7fe52016a5d0 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe5254b1700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe52013f600 0x7fe52015c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe5264b3700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52016a5d0 0x7fe520169880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe518013000 tx=0x7fe518019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe515ffb700 1 -- 172.21.15.123:0/2433831598 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51801b420 con 0x7fe52016a5d0 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe515ffb700 1 -- 172.21.15.123:0/2433831598 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe51801b5a0 con 0x7fe52016a5d0 2023-10-06T04:31:29.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5200c7da0 con 0x7fe52016a5d0 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe515ffb700 1 -- 172.21.15.123:0/2433831598 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe51801b960 con 0x7fe52016a5d0 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.178+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5200c77e0 con 0x7fe52016a5d0 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.179+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe52013dec0 con 0x7fe52016a5d0 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.180+0000 7fe515ffb700 1 -- 172.21.15.123:0/2433831598 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe51801bd50 con 0x7fe52016a5d0 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.180+0000 7fe515ffb700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.180+0000 7fe515ffb700 1 -- 172.21.15.123:0/2433831598 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fe518086070 con 0x7fe52016a5d0 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.180+0000 7fe525cb2700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.180+0000 7fe525cb2700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe504066f30 0x7fe5040693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe51c007fd0 tx=0x7fe51c007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.184+0000 7fe515ffb700 1 -- 172.21.15.123:0/2433831598 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe518053550 con 0x7fe52016a5d0 2023-10-06T04:31:29.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.369+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe520068d30 con 0x7fe52016a5d0 2023-10-06T04:31:29.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.370+0000 7fe515ffb700 1 -- 172.21.15.123:0/2433831598 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v18) v1 ==== 74+0+13804 (secure 0 0 0) 0x7fe518056af0 con 0x7fe52016a5d0 2023-10-06T04:31:29.371 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:29.371 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":18,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:29.007206+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:29.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe504066f30 msgr2=0x7fe5040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe504066f30 0x7fe5040693e0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe51c007fd0 tx=0x7fe51c007d10 comp rx=0 tx=0).stop 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52016a5d0 msgr2=0x7fe520169880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52016a5d0 0x7fe520169880 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe518013000 tx=0x7fe518019000 comp rx=0 tx=0).stop 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 shutdown_connections 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe504066f30 0x7fe5040693e0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe52016a5d0 0x7fe520169880 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe52006bb60 0x7fe520155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 --2- 172.21.15.123:0/2433831598 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe52013f600 0x7fe52015c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 >> 172.21.15.123:0/2433831598 conn(0x7fe5200ce0c0 msgr2=0x7fe52015cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:29.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 shutdown_connections 2023-10-06T04:31:29.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.372+0000 7fe527f16700 1 -- 172.21.15.123:0/2433831598 wait complete. 2023-10-06T04:31:29.385 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 32769, 'flags_names': 'hashpspool,creating', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '18', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T04:31:29.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:31:29.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.505+0000 7f672d438700 1 Processor -- start 2023-10-06T04:31:29.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.505+0000 7f672d438700 1 -- start start 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.505+0000 7f672d438700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6728159650 0x7f67280cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f672d438700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cf110 0x7f6728163240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f672d438700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67280cc850 con 0x7f67280cb630 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f672d438700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67280cbc40 con 0x7f6728159650 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f672d438700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67280cbf20 con 0x7f67280cf110 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f67277fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cf110 0x7f6728163240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6726ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6728159650 0x7f67280cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f67267fc700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f67280cb630 0x7f67280cea00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:46682/0 (socket says 172.21.15.123:46682) 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f67267fc700 1 -- 172.21.15.123:0/2880195840 learned_addr learned my addr 172.21.15.123:0/2880195840 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:29.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6726ffd700 1 -- 172.21.15.123:0/2880195840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cf110 msgr2=0x7f6728163240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6726ffd700 1 --2- 172.21.15.123:0/2880195840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cf110 0x7f6728163240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6726ffd700 1 -- 172.21.15.123:0/2880195840 >> v1:172.21.15.177:6789/0 conn(0x7f67280cb630 legacy=0x7f67280cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6726ffd700 1 -- 172.21.15.123:0/2880195840 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67280cc200 con 0x7f6728159650 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6725ffb700 1 -- 172.21.15.123:0/2880195840 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 666583749 0 0) 0x7f67280cc850 con 0x7f67280cb630 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6726ffd700 1 --2- 172.21.15.123:0/2880195840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 0x7f67280cb0f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67100098d0 tx=0x7f6710009120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a335aa0da38ac501 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.506+0000 7f6725ffb700 1 -- 172.21.15.123:0/2880195840 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710005c10 con 0x7f6728159650 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f6725ffb700 1 -- 172.21.15.123:0/2880195840 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6710005d70 con 0x7f6728159650 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f6725ffb700 1 -- 172.21.15.123:0/2880195840 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6710006280 con 0x7f6728159650 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 -- 172.21.15.123:0/2880195840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 msgr2=0x7f67280cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 --2- 172.21.15.123:0/2880195840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 0x7f67280cb0f0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f67100098d0 tx=0x7f6710009120 comp rx=0 tx=0).stop 2023-10-06T04:31:29.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 -- 172.21.15.123:0/2880195840 shutdown_connections 2023-10-06T04:31:29.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 --2- 172.21.15.123:0/2880195840 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cf110 0x7f6728163240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 --2- 172.21.15.123:0/2880195840 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 0x7f67280cb0f0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 -- 172.21.15.123:0/2880195840 >> 172.21.15.123:0/2880195840 conn(0x7f6728099610 msgr2=0x7f6728063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:29.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 -- 172.21.15.123:0/2880195840 shutdown_connections 2023-10-06T04:31:29.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.507+0000 7f672d438700 1 -- 172.21.15.123:0/2880195840 wait complete. 2023-10-06T04:31:29.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.508+0000 7f672d438700 1 Processor -- start 2023-10-06T04:31:29.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.508+0000 7f672d438700 1 -- start start 2023-10-06T04:31:29.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f672d438700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cb630 0x7f672815e530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f672d438700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67280cf110 0x7f672816ce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f672d438700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 0x7f672816d340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f672d438700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67281589d0 con 0x7f67280cf110 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f672d438700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6728157dc0 con 0x7f6728159650 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f672d438700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67281580a0 con 0x7f67280cb630 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f67267fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67280cf110 0x7f672816ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f67267fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67280cf110 0x7f672816ce00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45682/0 (socket says 172.21.15.123:45682) 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f67267fc700 1 -- 172.21.15.123:0/717076017 learned_addr learned my addr 172.21.15.123:0/717076017 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f67277fe700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 0x7f672816d340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.509+0000 7f6726ffd700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cb630 0x7f672815e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f67267fc700 1 -- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cb630 msgr2=0x7f672815e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f67267fc700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cb630 0x7f672815e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f67267fc700 1 -- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 msgr2=0x7f672816d340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f67267fc700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 0x7f672816d340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f67267fc700 1 -- 172.21.15.123:0/717076017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67100093c0 con 0x7f67280cf110 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f67267fc700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67280cf110 0x7f672816ce00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f671800c7e0 tx=0x7f6718007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f671effd700 1 -- 172.21.15.123:0/717076017 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718005b30 con 0x7f67280cf110 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f671effd700 1 -- 172.21.15.123:0/717076017 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6718005c90 con 0x7f67280cf110 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6728158380 con 0x7f67280cf110 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f671effd700 1 -- 172.21.15.123:0/717076017 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6718007300 con 0x7f67280cf110 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.510+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f672815d830 con 0x7f67280cf110 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.511+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6728158b80 con 0x7f67280cf110 2023-10-06T04:31:29.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.511+0000 7f671effd700 1 -- 172.21.15.123:0/717076017 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f671801b3b0 con 0x7f67280cf110 2023-10-06T04:31:29.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.512+0000 7f671effd700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6708066fb0 0x7f6708069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.512+0000 7f671effd700 1 -- 172.21.15.123:0/717076017 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6718083900 con 0x7f67280cf110 2023-10-06T04:31:29.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.512+0000 7f6726ffd700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6708066fb0 0x7f6708069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.512+0000 7f6726ffd700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6708066fb0 0x7f6708069460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67280ca460 tx=0x7f6710016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.516+0000 7f671effd700 1 -- 172.21.15.123:0/717076017 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6728158b80 con 0x7f67280cf110 2023-10-06T04:31:29.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.695+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6728158b10 con 0x7f67280cf110 2023-10-06T04:31:29.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.696+0000 7f671effd700 1 -- 172.21.15.123:0/717076017 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v18) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6718017020 con 0x7f67280cf110 2023-10-06T04:31:29.697 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:31:29.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6708066fb0 msgr2=0x7f6708069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6708066fb0 0x7f6708069460 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f67280ca460 tx=0x7f6710016000 comp rx=0 tx=0).stop 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67280cf110 msgr2=0x7f672816ce00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67280cf110 0x7f672816ce00 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f671800c7e0 tx=0x7f6718007df0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 shutdown_connections 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6708066fb0 0x7f6708069460 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6728159650 0x7f672816d340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f67280cb630 0x7f672815e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 --2- 172.21.15.123:0/717076017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f67280cf110 0x7f672816ce00 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 >> 172.21.15.123:0/717076017 conn(0x7f6728099610 msgr2=0x7f67280c56f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 shutdown_connections 2023-10-06T04:31:29.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.698+0000 7f672d438700 1 -- 172.21.15.123:0/717076017 wait complete. 2023-10-06T04:31:29.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:31:29.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.829+0000 7f87e88e3700 1 Processor -- start 2023-10-06T04:31:29.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.829+0000 7f87e88e3700 1 -- start start 2023-10-06T04:31:29.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.830+0000 7f87e88e3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f87e415a1e0 0x7f87e406a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.830+0000 7f87e88e3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e406af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.830+0000 7f87e88e3700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87e406b9c0 con 0x7f87e406b4d0 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.830+0000 7f87e88e3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87e4069aa0 con 0x7f87e415a1e0 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.830+0000 7f87e88e3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4069d80 con 0x7f87e415a5b0 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e406af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e259c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f87e415a1e0 0x7f87e406a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e406af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:42266/0 (socket says 172.21.15.123:42266) 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e2d9d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f87e406b4d0 0x7f87e4068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:46688/0 (socket says 172.21.15.123:46688) 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 -- 172.21.15.123:0/3837948881 learned_addr learned my addr 172.21.15.123:0/3837948881 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e259c700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f87e415a1e0 0x7f87e406a860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:58546/0 (socket says 172.21.15.123:58546) 2023-10-06T04:31:29.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 -- 172.21.15.123:0/3837948881 >> v2:172.21.15.177:3300/0 conn(0x7f87e415a1e0 msgr2=0x7f87e406a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 --2- 172.21.15.123:0/3837948881 >> v2:172.21.15.177:3300/0 conn(0x7f87e415a1e0 0x7f87e406a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 -- 172.21.15.123:0/3837948881 >> v1:172.21.15.177:6789/0 conn(0x7f87e406b4d0 legacy=0x7f87e4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 -- 172.21.15.123:0/3837948881 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87e406a060 con 0x7f87e415a5b0 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e259c700 1 --2- 172.21.15.123:0/3837948881 >> v2:172.21.15.177:3300/0 conn(0x7f87e415a1e0 0x7f87e406a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e1d9b700 1 --2- 172.21.15.123:0/3837948881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e406af90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87d40099b0 tx=0x7f87d4009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c3ff41884654e2d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e159a700 1 -- 172.21.15.123:0/3837948881 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d4009ea0 con 0x7f87e415a5b0 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e159a700 1 -- 172.21.15.123:0/3837948881 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d400b860 con 0x7f87e415a5b0 2023-10-06T04:31:29.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.831+0000 7f87e159a700 1 -- 172.21.15.123:0/3837948881 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400bc00 con 0x7f87e415a5b0 2023-10-06T04:31:29.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 -- 172.21.15.123:0/3837948881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 msgr2=0x7f87e406af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 --2- 172.21.15.123:0/3837948881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e406af90 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f87d40099b0 tx=0x7f87d4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:29.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 -- 172.21.15.123:0/3837948881 shutdown_connections 2023-10-06T04:31:29.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 --2- 172.21.15.123:0/3837948881 >> v2:172.21.15.177:3300/0 conn(0x7f87e415a1e0 0x7f87e406a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 --2- 172.21.15.123:0/3837948881 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e406af90 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 -- 172.21.15.123:0/3837948881 >> 172.21.15.123:0/3837948881 conn(0x7f87e40c9ca0 msgr2=0x7f87e406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 -- 172.21.15.123:0/3837948881 shutdown_connections 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.832+0000 7f87e88e3700 1 -- 172.21.15.123:0/3837948881 wait complete. 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.833+0000 7f87e88e3700 1 Processor -- start 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.833+0000 7f87e88e3700 1 -- start start 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e88e3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e406b4d0 0x7f87e4155c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e88e3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87e415a1e0 0x7f87e40c7bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e88e3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e40a54a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e88e3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e40c8530 con 0x7f87e415a1e0 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e88e3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87e4169eb0 con 0x7f87e406b4d0 2023-10-06T04:31:29.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e88e3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87e416a190 con 0x7f87e415a5b0 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87e415a1e0 0x7f87e40c7bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87e415a1e0 0x7f87e40c7bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45690/0 (socket says 172.21.15.123:45690) 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 -- 172.21.15.123:0/1273299092 learned_addr learned my addr 172.21.15.123:0/1273299092 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e2d9d700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e40a54a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e259c700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e406b4d0 0x7f87e4155c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 -- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 msgr2=0x7f87e40a54a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e40a54a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 -- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e406b4d0 msgr2=0x7f87e4155c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e406b4d0 0x7f87e4155c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 -- 172.21.15.123:0/1273299092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87d8009420 con 0x7f87e415a1e0 2023-10-06T04:31:29.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e259c700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e406b4d0 0x7f87e4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e2d9d700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e40a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87e1d9b700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87e415a1e0 0x7f87e40c7bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87e406e660 tx=0x7f87d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.834+0000 7f87d27fc700 1 -- 172.21.15.123:0/1273299092 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d400bbe0 con 0x7f87e415a1e0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.835+0000 7f87d27fc700 1 -- 172.21.15.123:0/1273299092 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87d401a450 con 0x7f87e415a1e0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.835+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87d40094f0 con 0x7f87e415a1e0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.835+0000 7f87d27fc700 1 -- 172.21.15.123:0/1273299092 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87d401a7f0 con 0x7f87e415a1e0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.835+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87e4169ba0 con 0x7f87e415a1e0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.835+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87e406a340 con 0x7f87e415a1e0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.836+0000 7f87d27fc700 1 -- 172.21.15.123:0/1273299092 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f87d401ab60 con 0x7f87e415a1e0 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.836+0000 7f87d27fc700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:29.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.836+0000 7f87d27fc700 1 -- 172.21.15.123:0/1273299092 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f87d4085da0 con 0x7f87e415a1e0 2023-10-06T04:31:29.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.836+0000 7f87e259c700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:29.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.837+0000 7f87e259c700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f87c4066f30 0x7f87c40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87d80093f0 tx=0x7f87d8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:29.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:29.840+0000 7f87d27fc700 1 -- 172.21.15.123:0/1273299092 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87d4053200 con 0x7f87e415a1e0 2023-10-06T04:31:30.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.025+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f87e4159a60 con 0x7f87e415a1e0 2023-10-06T04:31:30.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.025+0000 7f87d27fc700 1 -- 172.21.15.123:0/1273299092 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f87d4056820 con 0x7f87e415a1e0 2023-10-06T04:31:30.027 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f87c4066f30 msgr2=0x7f87c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f87c4066f30 0x7f87c40693e0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f87d80093f0 tx=0x7f87d8008040 comp rx=0 tx=0).stop 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87e415a1e0 msgr2=0x7f87e40c7bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87e415a1e0 0x7f87e40c7bc0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f87e406e660 tx=0x7f87d4017000 comp rx=0 tx=0).stop 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 shutdown_connections 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f87c4066f30 0x7f87c40693e0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f87e415a5b0 0x7f87e40a54a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87e406b4d0 0x7f87e4155c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 --2- 172.21.15.123:0/1273299092 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87e415a1e0 0x7f87e40c7bc0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 >> 172.21.15.123:0/1273299092 conn(0x7f87e40c9ca0 msgr2=0x7f87e40acce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:30.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 shutdown_connections 2023-10-06T04:31:30.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.027+0000 7f87e88e3700 1 -- 172.21.15.123:0/1273299092 wait complete. 2023-10-06T04:31:30.040 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:31:30.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.159+0000 7fa4852e1700 1 Processor -- start 2023-10-06T04:31:30.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.159+0000 7fa4852e1700 1 -- start start 2023-10-06T04:31:30.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa4852e1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4800b05b0 0x7fa4800b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa4852e1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48015b0d0 con 0x7fa4800b0a20 2023-10-06T04:31:30.162 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa4852e1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa48015b370 con 0x7fa4801591b0 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa4852e1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa480159c90 con 0x7fa4800b05b0 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa47e7fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa4800b05b0 0x7fa4800b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa47f7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa4800b0a20 0x7fa4800cf380 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:59704/0 (socket says 172.21.15.123:59704) 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa47f7fe700 1 -- 172.21.15.123:0/1431299113 learned_addr learned my addr 172.21.15.123:0/1431299113 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa47e7fc700 1 -- 172.21.15.123:0/1431299113 >> v1:172.21.15.177:6789/0 conn(0x7fa4801591b0 legacy=0x7fa480159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa47e7fc700 1 -- 172.21.15.123:0/1431299113 >> v1:172.21.15.123:6789/0 conn(0x7fa4800b0a20 legacy=0x7fa4800cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.160+0000 7fa47e7fc700 1 -- 172.21.15.123:0/1431299113 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4800b0fe0 con 0x7fa4800b05b0 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa47e7fc700 1 --2- 172.21.15.123:0/1431299113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4800b05b0 0x7fa4800b3ca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4700098e0 tx=0x7fa4700091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fb8624185c9214 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa47dffb700 1 -- 172.21.15.123:0/1431299113 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa470009e80 con 0x7fa4800b05b0 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa47dffb700 1 -- 172.21.15.123:0/1431299113 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47000b860 con 0x7fa4800b05b0 2023-10-06T04:31:30.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa47dffb700 1 -- 172.21.15.123:0/1431299113 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47000bbe0 con 0x7fa4800b05b0 2023-10-06T04:31:30.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa4852e1700 1 -- 172.21.15.123:0/1431299113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4800b05b0 msgr2=0x7fa4800b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1431299113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4800b05b0 0x7fa4800b3ca0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fa4700098e0 tx=0x7fa4700091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa4852e1700 1 -- 172.21.15.123:0/1431299113 shutdown_connections 2023-10-06T04:31:30.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1431299113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4800b05b0 0x7fa4800b3ca0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa4852e1700 1 -- 172.21.15.123:0/1431299113 >> 172.21.15.123:0/1431299113 conn(0x7fa4800995a0 msgr2=0x7fa480063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:30.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa4852e1700 1 -- 172.21.15.123:0/1431299113 shutdown_connections 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.161+0000 7fa4852e1700 1 -- 172.21.15.123:0/1431299113 wait complete. 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.162+0000 7fa4852e1700 1 Processor -- start 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa4852e1700 1 -- start start 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa4852e1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4800b05b0 0x7fa48016c700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa4852e1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4800b0a20 0x7fa4800adcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa4852e1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4801591b0 0x7fa48009d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa4852e1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800af380 con 0x7fa4801591b0 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa4852e1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800ae700 con 0x7fa4800b05b0 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa4852e1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4800ae9e0 con 0x7fa4800b0a20 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4801591b0 0x7fa48009d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa47f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4801591b0 0x7fa48009d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:45698/0 (socket says 172.21.15.123:45698) 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa47f7fe700 1 -- 172.21.15.123:0/1412622617 learned_addr learned my addr 172.21.15.123:0/1412622617 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa47effd700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4800b05b0 0x7fa48016c700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.163+0000 7fa47e7fc700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4800b0a20 0x7fa4800adcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa47f7fe700 1 -- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4800b0a20 msgr2=0x7fa4800adcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa47f7fe700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4800b0a20 0x7fa4800adcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa47f7fe700 1 -- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4800b05b0 msgr2=0x7fa48016c700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa47f7fe700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4800b05b0 0x7fa48016c700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa47f7fe700 1 -- 172.21.15.123:0/1412622617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa470009420 con 0x7fa4801591b0 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa47effd700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4800b05b0 0x7fa48016c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:30.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa47f7fe700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4801591b0 0x7fa48009d470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa47800cc40 tx=0x7fa478001a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa466ffd700 1 -- 172.21.15.123:0/1412622617 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa47800a9c0 con 0x7fa4801591b0 2023-10-06T04:31:30.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa466ffd700 1 -- 172.21.15.123:0/1412622617 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa47800ab20 con 0x7fa4801591b0 2023-10-06T04:31:30.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4800aecc0 con 0x7fa4801591b0 2023-10-06T04:31:30.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa466ffd700 1 -- 172.21.15.123:0/1412622617 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4780070f0 con 0x7fa4801591b0 2023-10-06T04:31:30.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.164+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa48009ef90 con 0x7fa4801591b0 2023-10-06T04:31:30.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.165+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa480169ac0 con 0x7fa4801591b0 2023-10-06T04:31:30.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.165+0000 7fa466ffd700 1 -- 172.21.15.123:0/1412622617 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa4780074e0 con 0x7fa4801591b0 2023-10-06T04:31:30.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.166+0000 7fa466ffd700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa46c066f30 0x7fa46c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.166+0000 7fa47effd700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa46c066f30 0x7fa46c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.166+0000 7fa466ffd700 1 -- 172.21.15.123:0/1412622617 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fa478084760 con 0x7fa4801591b0 2023-10-06T04:31:30.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.166+0000 7fa47effd700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa46c066f30 0x7fa46c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa474007fd0 tx=0x7fa474009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.170+0000 7fa466ffd700 1 -- 172.21.15.123:0/1412622617 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa478051c40 con 0x7fa4801591b0 2023-10-06T04:31:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.392+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa4800b12c0 con 0x7fa4801591b0 2023-10-06T04:31:30.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.393+0000 7fa466ffd700 1 -- 172.21.15.123:0/1412622617 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3533 (secure 0 0 0) 0x7fa478055260 con 0x7fa4801591b0 2023-10-06T04:31:30.395 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:30.395 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4293,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa46c066f30 msgr2=0x7fa46c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa46c066f30 0x7fa46c0693e0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa474007fd0 tx=0x7fa474009040 comp rx=0 tx=0).stop 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4801591b0 msgr2=0x7fa48009d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4801591b0 0x7fa48009d470 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fa47800cc40 tx=0x7fa478001a50 comp rx=0 tx=0).stop 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 shutdown_connections 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa46c066f30 0x7fa46c0693e0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4801591b0 0x7fa48009d470 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa4800b05b0 0x7fa48016c700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 --2- 172.21.15.123:0/1412622617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa4800b0a20 0x7fa4800adcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 >> 172.21.15.123:0/1412622617 conn(0x7fa4800995a0 msgr2=0x7fa480068380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 shutdown_connections 2023-10-06T04:31:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.395+0000 7fa4852e1700 1 -- 172.21.15.123:0/1412622617 wait complete. 2023-10-06T04:31:30.398 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 3 2023-10-06T04:31:30.409 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:30.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.524+0000 7fcfd86dd700 1 Processor -- start 2023-10-06T04:31:30.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.524+0000 7fcfd86dd700 1 -- start start 2023-10-06T04:31:30.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd86dd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcfd0068d80 0x7fcfd00c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd86dd700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd00c7c50 con 0x7fcfd00c7460 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd86dd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd00c7f30 con 0x7fcfd0155780 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd86dd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcfd00c8210 con 0x7fcfd0068d80 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd5c78700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcfd0068d80 0x7fcfd00c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd6c7a700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcfd00c7460 0x7fcfd006aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33260/0 (socket says 172.21.15.123:33260) 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd6c7a700 1 -- 172.21.15.123:0/2794727007 learned_addr learned my addr 172.21.15.123:0/2794727007 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd5c78700 1 -- 172.21.15.123:0/2794727007 >> v1:172.21.15.177:6789/0 conn(0x7fcfd0155780 legacy=0x7fcfd0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd5c78700 1 -- 172.21.15.123:0/2794727007 >> v1:172.21.15.123:6789/0 conn(0x7fcfd00c7460 legacy=0x7fcfd006aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd5477700 1 -- 172.21.15.123:0/2794727007 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329255889 0 0) 0x7fcfd00c7c50 con 0x7fcfd00c7460 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.525+0000 7fcfd5c78700 1 -- 172.21.15.123:0/2794727007 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd00c84f0 con 0x7fcfd0068d80 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd5c78700 1 --2- 172.21.15.123:0/2794727007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd0068d80 0x7fcfd00c6f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0098e0 tx=0x7fcfcc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb31b9bd0e837db server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd5477700 1 -- 172.21.15.123:0/2794727007 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc009e80 con 0x7fcfd0068d80 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd5477700 1 -- 172.21.15.123:0/2794727007 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00b860 con 0x7fcfd0068d80 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd5477700 1 -- 172.21.15.123:0/2794727007 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc00bbe0 con 0x7fcfd0068d80 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd86dd700 1 -- 172.21.15.123:0/2794727007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd0068d80 msgr2=0x7fcfd00c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/2794727007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd0068d80 0x7fcfd00c6f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fcfcc0098e0 tx=0x7fcfcc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd86dd700 1 -- 172.21.15.123:0/2794727007 shutdown_connections 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/2794727007 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd0068d80 0x7fcfd00c6f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd86dd700 1 -- 172.21.15.123:0/2794727007 >> 172.21.15.123:0/2794727007 conn(0x7fcfd00c9c90 msgr2=0x7fcfd006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd86dd700 1 -- 172.21.15.123:0/2794727007 shutdown_connections 2023-10-06T04:31:30.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.526+0000 7fcfd86dd700 1 -- 172.21.15.123:0/2794727007 wait complete. 2023-10-06T04:31:30.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.527+0000 7fcfd86dd700 1 Processor -- start 2023-10-06T04:31:30.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.527+0000 7fcfd86dd700 1 -- start start 2023-10-06T04:31:30.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd86dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd0068d80 0x7fcfd0150eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd86dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd00c7460 0x7fcfd00ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd86dd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd0155780 0x7fcfd00caa50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd86dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00ad280 con 0x7fcfd00c7460 2023-10-06T04:31:30.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd86dd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00c4750 con 0x7fcfd0155780 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd86dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd00accc0 con 0x7fcfd0068d80 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd00c7460 0x7fcfd00ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd00c7460 0x7fcfd00ac200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52048/0 (socket says 172.21.15.123:52048) 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 -- 172.21.15.123:0/605931037 learned_addr learned my addr 172.21.15.123:0/605931037 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd6c7a700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd0155780 0x7fcfd00caa50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd6479700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd0068d80 0x7fcfd0150eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 -- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd0068d80 msgr2=0x7fcfd0150eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd0068d80 0x7fcfd0150eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 -- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd0155780 msgr2=0x7fcfd00caa50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd0155780 0x7fcfd00caa50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd5c78700 1 -- 172.21.15.123:0/605931037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfcc009420 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.528+0000 7fcfd6c7a700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd0155780 0x7fcfd00caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.529+0000 7fcfd5c78700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd00c7460 0x7fcfd00ac200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcfd006d230 tx=0x7fcfcc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.529+0000 7fcfc67fc700 1 -- 172.21.15.123:0/605931037 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc01f070 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.529+0000 7fcfc67fc700 1 -- 172.21.15.123:0/605931037 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfcc00b9d0 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.529+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd00ad000 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.529+0000 7fcfc67fc700 1 -- 172.21.15.123:0/605931037 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfcc025670 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.529+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd01513f0 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.529+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd0151ad0 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.530+0000 7fcfc67fc700 1 -- 172.21.15.123:0/605931037 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfcc023070 con 0x7fcfd00c7460 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.530+0000 7fcfc67fc700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfb4066f30 0x7fcfb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.530+0000 7fcfc67fc700 1 -- 172.21.15.123:0/605931037 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fcfcc0859d0 con 0x7fcfd00c7460 2023-10-06T04:31:30.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.530+0000 7fcfd6479700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfb4066f30 0x7fcfb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.531+0000 7fcfd6479700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfb4066f30 0x7fcfb40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00037c0 tx=0x7fcfc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.535+0000 7fcfc67fc700 1 -- 172.21.15.123:0/605931037 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfcc052e30 con 0x7fcfd00c7460 2023-10-06T04:31:30.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.716+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcfd00c8860 con 0x7fcfd00c7460 2023-10-06T04:31:30.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.716+0000 7fcfc67fc700 1 -- 172.21.15.123:0/605931037 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v19) v1 ==== 74+0+13791 (secure 0 0 0) 0x7fcfcc02a090 con 0x7fcfd00c7460 2023-10-06T04:31:30.718 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:30.718 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":19,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:30.012917+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:30.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.718+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfb4066f30 msgr2=0x7fcfb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.718+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfb4066f30 0x7fcfb40693e0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fcfc00037c0 tx=0x7fcfc0009040 comp rx=0 tx=0).stop 2023-10-06T04:31:30.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.718+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd00c7460 msgr2=0x7fcfd00ac200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.718+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd00c7460 0x7fcfd00ac200 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fcfd006d230 tx=0x7fcfcc017000 comp rx=0 tx=0).stop 2023-10-06T04:31:30.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 shutdown_connections 2023-10-06T04:31:30.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfb4066f30 0x7fcfb40693e0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcfd0155780 0x7fcfd00caa50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcfd0068d80 0x7fcfd0150eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 --2- 172.21.15.123:0/605931037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcfd00c7460 0x7fcfd00ac200 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 >> 172.21.15.123:0/605931037 conn(0x7fcfd00c9c90 msgr2=0x7fcfd0067660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:30.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 shutdown_connections 2023-10-06T04:31:30.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.719+0000 7fcfd86dd700 1 -- 172.21.15.123:0/605931037 wait complete. 2023-10-06T04:31:30.732 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T04:31:30.732 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:31:30.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.851+0000 7f68c9263700 1 Processor -- start 2023-10-06T04:31:30.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.851+0000 7f68c9263700 1 -- start start 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c9263700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f68c406e9f0 0x7f68c4161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c9263700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c406a4e0 con 0x7f68c413fa70 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c9263700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c40c8ec0 con 0x7f68c40c8a10 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c9263700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c40c91a0 con 0x7f68c406e9f0 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c37fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f68c406e9f0 0x7f68c4161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c2ffd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f68c413fa70 0x7f68c40c8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33272/0 (socket says 172.21.15.123:33272) 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c37fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f68c406e9f0 0x7f68c4161200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52060/0 (socket says 172.21.15.123:52060) 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c2ffd700 1 -- 172.21.15.123:0/4251238536 learned_addr learned my addr 172.21.15.123:0/4251238536 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c37fe700 1 -- 172.21.15.123:0/4251238536 >> v1:172.21.15.177:6789/0 conn(0x7f68c40c8a10 legacy=0x7f68c406e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:30.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c37fe700 1 -- 172.21.15.123:0/4251238536 >> v1:172.21.15.123:6789/0 conn(0x7f68c413fa70 legacy=0x7f68c40c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.852+0000 7f68c37fe700 1 -- 172.21.15.123:0/4251238536 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c40c9480 con 0x7f68c406e9f0 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c37fe700 1 --2- 172.21.15.123:0/4251238536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c406e9f0 0x7f68c4161200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009e60 tx=0x7f68bc009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25c12245be5e229d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c1ffb700 1 -- 172.21.15.123:0/4251238536 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc00e070 con 0x7f68c406e9f0 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c1ffb700 1 -- 172.21.15.123:0/4251238536 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc00bbf0 con 0x7f68c406e9f0 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c1ffb700 1 -- 172.21.15.123:0/4251238536 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc012430 con 0x7f68c406e9f0 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c9263700 1 -- 172.21.15.123:0/4251238536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c406e9f0 msgr2=0x7f68c4161200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c9263700 1 --2- 172.21.15.123:0/4251238536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c406e9f0 0x7f68c4161200 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f68bc009e60 tx=0x7f68bc009780 comp rx=0 tx=0).stop 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c9263700 1 -- 172.21.15.123:0/4251238536 shutdown_connections 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c9263700 1 --2- 172.21.15.123:0/4251238536 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c406e9f0 0x7f68c4161200 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c9263700 1 -- 172.21.15.123:0/4251238536 >> 172.21.15.123:0/4251238536 conn(0x7f68c40cd020 msgr2=0x7f68c40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c9263700 1 -- 172.21.15.123:0/4251238536 shutdown_connections 2023-10-06T04:31:30.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.853+0000 7f68c9263700 1 -- 172.21.15.123:0/4251238536 wait complete. 2023-10-06T04:31:30.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 Processor -- start 2023-10-06T04:31:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 -- start start 2023-10-06T04:31:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68c406e9f0 0x7f68c413cb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68c40c8a10 0x7f68c40aed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c413fa70 0x7f68c413e780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40aa380 con 0x7f68c413fa70 2023-10-06T04:31:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40aa660 con 0x7f68c40c8a10 2023-10-06T04:31:30.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c9263700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c40aa940 con 0x7f68c406e9f0 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c413fa70 0x7f68c413e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c413fa70 0x7f68c413e780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52076/0 (socket says 172.21.15.123:52076) 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.855+0000 7f68c37fe700 1 -- 172.21.15.123:0/587993391 learned_addr learned my addr 172.21.15.123:0/587993391 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c27fc700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68c40c8a10 0x7f68c40aed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c2ffd700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68c406e9f0 0x7f68c413cb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c37fe700 1 -- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68c406e9f0 msgr2=0x7f68c413cb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c37fe700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68c406e9f0 0x7f68c413cb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c37fe700 1 -- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68c40c8a10 msgr2=0x7f68c40aed70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c37fe700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68c40c8a10 0x7f68c40aed70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c37fe700 1 -- 172.21.15.123:0/587993391 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68bc009a20 con 0x7f68c413fa70 2023-10-06T04:31:30.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c2ffd700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68c406e9f0 0x7f68c413cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c37fe700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c413fa70 0x7f68c413e780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68bc003000 tx=0x7f68bc018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68aaffd700 1 -- 172.21.15.123:0/587993391 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc01e640 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68aaffd700 1 -- 172.21.15.123:0/587993391 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68bc01e7c0 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68c40aac20 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68aaffd700 1 -- 172.21.15.123:0/587993391 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68bc01eb80 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.856+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c413d060 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.857+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c406d240 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.857+0000 7f68aaffd700 1 -- 172.21.15.123:0/587993391 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f68bc024070 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.858+0000 7f68aaffd700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f68b0066f30 0x7f68b00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.858+0000 7f68aaffd700 1 -- 172.21.15.123:0/587993391 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f68bc086e20 con 0x7f68c413fa70 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.858+0000 7f68c2ffd700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f68b0066f30 0x7f68b00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:30.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.858+0000 7f68c2ffd700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f68b0066f30 0x7f68b00693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f68b8001820 tx=0x7f68b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:30.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:30.862+0000 7f68aaffd700 1 -- 172.21.15.123:0/587993391 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f68bc054280 con 0x7f68c413fa70 2023-10-06T04:31:31.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.045+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f68c404bb80 con 0x7f68c413fa70 2023-10-06T04:31:31.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.045+0000 7f68aaffd700 1 -- 172.21.15.123:0/587993391 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v19) v1 ==== 93+0+10 (secure 0 0 0) 0x7f68bc0578a0 con 0x7f68c413fa70 2023-10-06T04:31:31.046 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:31:31.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f68b0066f30 msgr2=0x7f68b00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f68b0066f30 0x7f68b00693e0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f68b8001820 tx=0x7f68b8009040 comp rx=0 tx=0).stop 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c413fa70 msgr2=0x7f68c413e780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c413fa70 0x7f68c413e780 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f68bc003000 tx=0x7f68bc018000 comp rx=0 tx=0).stop 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 shutdown_connections 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f68b0066f30 0x7f68b00693e0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f68c413fa70 0x7f68c413e780 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f68c406e9f0 0x7f68c413cb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 --2- 172.21.15.123:0/587993391 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f68c40c8a10 0x7f68c40aed70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 >> 172.21.15.123:0/587993391 conn(0x7f68c40cd020 msgr2=0x7f68c413d980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 shutdown_connections 2023-10-06T04:31:31.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.047+0000 7f68c9263700 1 -- 172.21.15.123:0/587993391 wait complete. 2023-10-06T04:31:31.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:31:31.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.178+0000 7f05acd8a700 1 Processor -- start 2023-10-06T04:31:31.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.178+0000 7f05acd8a700 1 -- start start 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.179+0000 7f05acd8a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 0x7f05a80ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.179+0000 7f05acd8a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a80c9d70 con 0x7f05a80c9150 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.179+0000 7f05acd8a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05a8159200 con 0x7f05a80caf00 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.179+0000 7f05acd8a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8157800 con 0x7f05a8157430 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.179+0000 7f05a6d9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f05a80c9150 0x7f05a8159690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33276/0 (socket says 172.21.15.123:33276) 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.179+0000 7f05a6d9d700 1 -- 172.21.15.123:0/4265612996 learned_addr learned my addr 172.21.15.123:0/4265612996 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a659c700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 0x7f05a80ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 919921292 0 0) 0x7f05a0001490 con 0x7f05a80c9150 2023-10-06T04:31:31.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0590003570 con 0x7f05a80c9150 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3993449975 0 0) 0x7f05a8159200 con 0x7f05a80caf00 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a0001490 con 0x7f05a80caf00 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3982615778 0 0) 0x7f0590003570 con 0x7f05a80c9150 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 msgr2=0x7f05a80ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 0x7f05a80ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 >> v1:172.21.15.177:6789/0 conn(0x7f05a80caf00 legacy=0x7f05a80c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80c97b0 con 0x7f05a80c9150 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f05a0001980 con 0x7f05a80c9150 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 >> v1:172.21.15.123:6789/0 conn(0x7f05a80c9150 legacy=0x7f05a8159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590003c20 0x7f0590004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590004810 0x7f0590004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 0x7f05900051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a8157800 con 0x7f0590004810 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f059000bb40 con 0x7f0590003c20 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f059000bd00 con 0x7f05a8157430 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a5d9b700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590004810 0x7f0590004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a659c700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590003c20 0x7f0590004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a6d9d700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 0x7f05900051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a5d9b700 1 -- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 msgr2=0x7f05900051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a5d9b700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a8157430 0x7f05900051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a5d9b700 1 -- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590003c20 msgr2=0x7f0590004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a5d9b700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590003c20 0x7f0590004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.180+0000 7f05a5d9b700 1 -- 172.21.15.123:0/4265612996 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0590003570 con 0x7f0590004810 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05a5d9b700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590004810 0x7f0590004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0598001a10 tx=0x7f0598007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8ebb7a713205625 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800c5d0 con 0x7f0590004810 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f059800c730 con 0x7f0590004810 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05a559a700 1 -- 172.21.15.123:0/4265612996 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f059800cad0 con 0x7f0590004810 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05acd8a700 1 -- 172.21.15.123:0/4265612996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590004810 msgr2=0x7f0590004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05acd8a700 1 --2- 172.21.15.123:0/4265612996 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0590004810 0x7f0590004c80 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f0598001a10 tx=0x7f0598007d30 comp rx=0 tx=0).stop 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05a6d9d700 1 -- 172.21.15.123:0/4265612996 reap_dead start 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05acd8a700 1 -- 172.21.15.123:0/4265612996 shutdown_connections 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05acd8a700 1 -- 172.21.15.123:0/4265612996 >> 172.21.15.123:0/4265612996 conn(0x7f05a8097450 msgr2=0x7f05a8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05acd8a700 1 -- 172.21.15.123:0/4265612996 shutdown_connections 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.181+0000 7f05acd8a700 1 -- 172.21.15.123:0/4265612996 wait complete. 2023-10-06T04:31:31.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.182+0000 7f05acd8a700 1 Processor -- start 2023-10-06T04:31:31.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.182+0000 7f05acd8a700 1 -- start start 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05acd8a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590004810 0x7f05a81536a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05acd8a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a80cd2b0 0x7f05a80c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05acd8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80c3cf0 0x7f05a80c4190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05acd8a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80c46d0 con 0x7f05a80c3cf0 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05acd8a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80ce7c0 con 0x7f0590004810 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05acd8a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05a80cd720 con 0x7f05a80cd2b0 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05a6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80c3cf0 0x7f05a80c4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05a6d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80c3cf0 0x7f05a80c4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52104/0 (socket says 172.21.15.123:52104) 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05a6d9d700 1 -- 172.21.15.123:0/4054291986 learned_addr learned my addr 172.21.15.123:0/4054291986 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:31.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05a5d9b700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a80cd2b0 0x7f05a80c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.183+0000 7f05a659c700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590004810 0x7f05a81536a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05a6d9d700 1 -- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a80cd2b0 msgr2=0x7f05a80c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05a6d9d700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a80cd2b0 0x7f05a80c6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05a6d9d700 1 -- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590004810 msgr2=0x7f05a81536a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05a6d9d700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590004810 0x7f05a81536a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05a6d9d700 1 -- 172.21.15.123:0/4054291986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a80cda00 con 0x7f05a80c3cf0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05a6d9d700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80c3cf0 0x7f05a80c4190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cf1b0 tx=0x7f05a000a0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05967fc700 1 -- 172.21.15.123:0/4054291986 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0001980 con 0x7f05a80c3cf0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05967fc700 1 -- 172.21.15.123:0/4054291986 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a00079b0 con 0x7f05a80c3cf0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05a80cdce0 con 0x7f05a80c3cf0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05967fc700 1 -- 172.21.15.123:0/4054291986 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a0013640 con 0x7f05a80c3cf0 2023-10-06T04:31:31.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.184+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05a80ce2a0 con 0x7f05a80c3cf0 2023-10-06T04:31:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.185+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05a809bd40 con 0x7f05a80c3cf0 2023-10-06T04:31:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.185+0000 7f05967fc700 1 -- 172.21.15.123:0/4054291986 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f05a0013a70 con 0x7f05a80c3cf0 2023-10-06T04:31:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.186+0000 7f05967fc700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0588066f30 0x7f05880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.186+0000 7f05967fc700 1 -- 172.21.15.123:0/4054291986 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f05a0084bd0 con 0x7f05a80c3cf0 2023-10-06T04:31:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.186+0000 7f05a659c700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0588066f30 0x7f05880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.186+0000 7f05a659c700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0588066f30 0x7f05880693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f059c006ea0 tx=0x7f059c009120 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.190+0000 7f05967fc700 1 -- 172.21.15.123:0/4054291986 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f05a0051f90 con 0x7f05a80c3cf0 2023-10-06T04:31:31.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.370+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f05a80c9d70 con 0x7f05a80c3cf0 2023-10-06T04:31:31.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.370+0000 7f05967fc700 1 -- 172.21.15.123:0/4054291986 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v19) v1 ==== 92+0+10 (secure 0 0 0) 0x7f05a80c9d70 con 0x7f05a80c3cf0 2023-10-06T04:31:31.372 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0588066f30 msgr2=0x7f05880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0588066f30 0x7f05880693e0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f059c006ea0 tx=0x7f059c009120 comp rx=0 tx=0).stop 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80c3cf0 msgr2=0x7f05a80c4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80c3cf0 0x7f05a80c4190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f05a80cf1b0 tx=0x7f05a000a0f0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 shutdown_connections 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0588066f30 0x7f05880693e0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f05a80c3cf0 0x7f05a80c4190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0590004810 0x7f05a81536a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 --2- 172.21.15.123:0/4054291986 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f05a80cd2b0 0x7f05a80c6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.372+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 >> 172.21.15.123:0/4054291986 conn(0x7f05a8097450 msgr2=0x7f05a8154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:31.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.373+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 shutdown_connections 2023-10-06T04:31:31.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.373+0000 7f05acd8a700 1 -- 172.21.15.123:0/4054291986 wait complete. 2023-10-06T04:31:31.386 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:31:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.501+0000 7f88ac353700 1 Processor -- start 2023-10-06T04:31:31.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.501+0000 7f88ac353700 1 -- start start 2023-10-06T04:31:31.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88ac353700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a40688c0 0x7f88a40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88ac353700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f88a4068c90 0x7f88a406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88ac353700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f88a406e970 0x7f88a4158f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88ac353700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a4069a30 con 0x7f88a4068c90 2023-10-06T04:31:31.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88ac353700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88a406a890 con 0x7f88a406e970 2023-10-06T04:31:31.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88ac353700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4069c80 con 0x7f88a40688c0 2023-10-06T04:31:31.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f88a4068c90 0x7f88a406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f88a4068c90 0x7f88a406e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52110/0 (socket says 172.21.15.123:52110) 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 -- 172.21.15.123:0/2659753543 learned_addr learned my addr 172.21.15.123:0/2659753543 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88aa0ef700 1 --2- 172.21.15.123:0/2659753543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a40688c0 0x7f88a40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88aa8f0700 1 --2- 172.21.15.123:0/2659753543 >> v2:172.21.15.177:3300/0 conn(0x7f88a406e970 0x7f88a4158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 -- 172.21.15.123:0/2659753543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a40688c0 msgr2=0x7f88a40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 --2- 172.21.15.123:0/2659753543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a40688c0 0x7f88a40c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 -- 172.21.15.123:0/2659753543 >> v2:172.21.15.177:3300/0 conn(0x7f88a406e970 msgr2=0x7f88a4158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 --2- 172.21.15.123:0/2659753543 >> v2:172.21.15.177:3300/0 conn(0x7f88a406e970 0x7f88a4158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 -- 172.21.15.123:0/2659753543 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88a4069f60 con 0x7f88a4068c90 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a98ee700 1 --2- 172.21.15.123:0/2659753543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a4068c90 0x7f88a406e430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a40cc6a0 tx=0x7f8894009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f3cb6451904e57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a90ed700 1 -- 172.21.15.123:0/2659753543 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889400b860 con 0x7f88a4068c90 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a90ed700 1 -- 172.21.15.123:0/2659753543 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f889400b9c0 con 0x7f88a4068c90 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.502+0000 7f88a90ed700 1 -- 172.21.15.123:0/2659753543 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f889400bd60 con 0x7f88a4068c90 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 -- 172.21.15.123:0/2659753543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a4068c90 msgr2=0x7f88a406e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 --2- 172.21.15.123:0/2659753543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a4068c90 0x7f88a406e430 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f88a40cc6a0 tx=0x7f8894009250 comp rx=0 tx=0).stop 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 -- 172.21.15.123:0/2659753543 shutdown_connections 2023-10-06T04:31:31.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 --2- 172.21.15.123:0/2659753543 >> v2:172.21.15.177:3300/0 conn(0x7f88a406e970 0x7f88a4158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 --2- 172.21.15.123:0/2659753543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a40688c0 0x7f88a40c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 --2- 172.21.15.123:0/2659753543 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a4068c90 0x7f88a406e430 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 -- 172.21.15.123:0/2659753543 >> 172.21.15.123:0/2659753543 conn(0x7f88a40c9c90 msgr2=0x7f88a406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:31.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 -- 172.21.15.123:0/2659753543 shutdown_connections 2023-10-06T04:31:31.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.503+0000 7f88ac353700 1 -- 172.21.15.123:0/2659753543 wait complete. 2023-10-06T04:31:31.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.504+0000 7f88ac353700 1 Processor -- start 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.504+0000 7f88ac353700 1 -- start start 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88ac353700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a40688c0 0x7f88a40c5860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88ac353700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a4068c90 0x7f88a4158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88ac353700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a406e970 0x7f88a4159410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88ac353700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a4143d20 con 0x7f88a40688c0 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88ac353700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40c47c0 con 0x7f88a4068c90 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88ac353700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88a40c4aa0 con 0x7f88a406e970 2023-10-06T04:31:31.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a40688c0 0x7f88a40c5860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a40688c0 0x7f88a40c5860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52118/0 (socket says 172.21.15.123:52118) 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 -- 172.21.15.123:0/1135286366 learned_addr learned my addr 172.21.15.123:0/1135286366 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88a98ee700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a4068c90 0x7f88a4158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa8f0700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a406e970 0x7f88a4159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 -- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a406e970 msgr2=0x7f88a4159410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a406e970 0x7f88a4159410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 -- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a4068c90 msgr2=0x7f88a4158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a4068c90 0x7f88a4158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 -- 172.21.15.123:0/1135286366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88940094f0 con 0x7f88a40688c0 2023-10-06T04:31:31.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa8f0700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a406e970 0x7f88a4159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:31.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f88aa0ef700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a40688c0 0x7f88a40c5860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88a000d890 tx=0x7f88a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.505+0000 7f889a7fc700 1 -- 172.21.15.123:0/1135286366 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a00125f0 con 0x7f88a40688c0 2023-10-06T04:31:31.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.506+0000 7f889a7fc700 1 -- 172.21.15.123:0/1135286366 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88a0012750 con 0x7f88a40688c0 2023-10-06T04:31:31.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.506+0000 7f889a7fc700 1 -- 172.21.15.123:0/1135286366 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88a0012ad0 con 0x7f88a40688c0 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.506+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88a40c4d80 con 0x7f88a40688c0 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.506+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88a40c5340 con 0x7f88a40688c0 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.506+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f88a40c56b0 con 0x7f88a40688c0 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.507+0000 7f889a7fc700 1 -- 172.21.15.123:0/1135286366 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f88a001e460 con 0x7f88a40688c0 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.507+0000 7f889a7fc700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8888066f30 0x7f88880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.507+0000 7f889a7fc700 1 -- 172.21.15.123:0/1135286366 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f88a0084960 con 0x7f88a40688c0 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.507+0000 7f88a98ee700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8888066f30 0x7f88880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.508+0000 7f88a98ee700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8888066f30 0x7f88880693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88a40cc6a0 tx=0x7f8894017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.511+0000 7f889a7fc700 1 -- 172.21.15.123:0/1135286366 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f88a0051dc0 con 0x7f88a40688c0 2023-10-06T04:31:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.694+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f88a406a240 con 0x7f88a40688c0 2023-10-06T04:31:31.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.695+0000 7f889a7fc700 1 -- 172.21.15.123:0/1135286366 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v19) v1 ==== 84+0+126 (secure 0 0 0) 0x7f88a00553e0 con 0x7f88a40688c0 2023-10-06T04:31:31.696 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:31.696 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2023-10-06T04:31:31.696 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:31:31.696 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2023-10-06T04:31:31.697 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2023-10-06T04:31:31.697 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:31.697 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:31:31.697 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 2, 2023-10-06T04:31:31.697 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": "rbd" 2023-10-06T04:31:31.697 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:31:31.697 INFO:teuthology.orchestra.run.smithi123.stdout:] 2023-10-06T04:31:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8888066f30 msgr2=0x7f88880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8888066f30 0x7f88880693e0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88a40cc6a0 tx=0x7f8894017000 comp rx=0 tx=0).stop 2023-10-06T04:31:31.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a40688c0 msgr2=0x7f88a40c5860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a40688c0 0x7f88a40c5860 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f88a000d890 tx=0x7f88a0008040 comp rx=0 tx=0).stop 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 shutdown_connections 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8888066f30 0x7f88880693e0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f88a40688c0 0x7f88a40c5860 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f88a406e970 0x7f88a4159410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 --2- 172.21.15.123:0/1135286366 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f88a4068c90 0x7f88a4158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 >> 172.21.15.123:0/1135286366 conn(0x7f88a40c9c90 msgr2=0x7f88a40c6ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 shutdown_connections 2023-10-06T04:31:31.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.697+0000 7f88ac353700 1 -- 172.21.15.123:0/1135286366 wait complete. 2023-10-06T04:31:31.710 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:31:31.710 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:31:31.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.831+0000 7f6b38ad2700 1 Processor -- start 2023-10-06T04:31:31.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.831+0000 7f6b38ad2700 1 -- start start 2023-10-06T04:31:31.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.832+0000 7f6b38ad2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6b34156220 0x7f6b3415a680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.832+0000 7f6b38ad2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340634b0 con 0x7f6b340cb650 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.832+0000 7f6b38ad2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340c9aa0 con 0x7f6b340aec90 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.832+0000 7f6b38ad2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b340c9d80 con 0x7f6b34156220 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6b340cb650 0x7f6b34067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33278/0 (socket says 172.21.15.123:33278) 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 -- 172.21.15.123:0/1513514075 learned_addr learned my addr 172.21.15.123:0/1513514075 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b32d9d700 1 --2- 172.21.15.123:0/1513514075 >> v2:172.21.15.177:3300/0 conn(0x7f6b34156220 0x7f6b3415a680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4248255954 0 0) 0x7f6b340634b0 con 0x7f6b340cb650 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b20003570 con 0x7f6b340cb650 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2852778288 0 0) 0x7f6b20003570 con 0x7f6b340cb650 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 >> v2:172.21.15.177:3300/0 conn(0x7f6b34156220 msgr2=0x7f6b3415a680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 --2- 172.21.15.123:0/1513514075 >> v2:172.21.15.177:3300/0 conn(0x7f6b34156220 0x7f6b3415a680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 >> v1:172.21.15.177:6789/0 conn(0x7f6b340aec90 legacy=0x7f6b34155b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340c7cc0 con 0x7f6b340cb650 2023-10-06T04:31:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f6b24001470 con 0x7f6b340cb650 2023-10-06T04:31:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 >> v1:172.21.15.123:6789/0 conn(0x7f6b340cb650 legacy=0x7f6b34067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b20003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b20004810 0x7f6b20004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b200051c0 0x7f6b2000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340c9d80 con 0x7f6b20003a60 2023-10-06T04:31:31.841 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2000c0d0 con 0x7f6b20004810 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b2000c290 con 0x7f6b200051c0 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b20003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b31d9b700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b20004810 0x7f6b20004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b32d9d700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b200051c0 0x7f6b2000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 -- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b200051c0 msgr2=0x7f6b2000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b200051c0 0x7f6b2000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 -- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b20004810 msgr2=0x7f6b20004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b20004810 0x7f6b20004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b3259c700 1 -- 172.21.15.123:0/1513514075 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b20003570 con 0x7f6b20003a60 2023-10-06T04:31:31.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b32d9d700 1 -- 172.21.15.123:0/1513514075 reap_dead start 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.833+0000 7f6b31d9b700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b20004810 0x7f6b20004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b3259c700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b20003e70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c390 tx=0x7f6b34067d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da234a231d270902 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400cbe0 con 0x7f6b20003a60 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400cd40 con 0x7f6b20003a60 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b3159a700 1 -- 172.21.15.123:0/1513514075 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24005b70 con 0x7f6b20003a60 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1513514075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 msgr2=0x7f6b20003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b20003e70 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f6b2400c390 tx=0x7f6b34067d70 comp rx=0 tx=0).stop 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1513514075 shutdown_connections 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1513514075 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b20003e70 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1513514075 >> 172.21.15.123:0/1513514075 conn(0x7f6b340cd250 msgr2=0x7f6b340c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:31.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1513514075 shutdown_connections 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.834+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1513514075 wait complete. 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.835+0000 7f6b38ad2700 1 Processor -- start 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.835+0000 7f6b38ad2700 1 -- start start 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b38ad2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b341584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b38ad2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b34156720 0x7f6b34156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b38ad2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b34167a20 0x7f6b34167ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b38ad2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34157430 con 0x7f6b20003a60 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b38ad2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b34157710 con 0x7f6b34156720 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b38ad2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b341579f0 con 0x7f6b34167a20 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b3259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b341584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b3259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b341584d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52134/0 (socket says 172.21.15.123:52134) 2023-10-06T04:31:31.844 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b3259c700 1 -- 172.21.15.123:0/1666193998 learned_addr learned my addr 172.21.15.123:0/1666193998 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b3259c700 1 -- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b34167a20 msgr2=0x7f6b34167ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b31d9b700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b34156720 0x7f6b34156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b32d9d700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b34167a20 0x7f6b34167ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b3259c700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b34167a20 0x7f6b34167ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b3259c700 1 -- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b34156720 msgr2=0x7f6b34156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.836+0000 7f6b3259c700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b34156720 0x7f6b34156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.837+0000 7f6b3259c700 1 -- 172.21.15.123:0/1666193998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b34157cd0 con 0x7f6b20003a60 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.837+0000 7f6b3259c700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b341584d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c0a0 tx=0x7f6b24015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.837+0000 7f6b2a7fc700 1 -- 172.21.15.123:0/1666193998 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b24022070 con 0x7f6b20003a60 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.837+0000 7f6b2a7fc700 1 -- 172.21.15.123:0/1666193998 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2400ceb0 con 0x7f6b20003a60 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.837+0000 7f6b2a7fc700 1 -- 172.21.15.123:0/1666193998 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2400cbe0 con 0x7f6b20003a60 2023-10-06T04:31:31.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.837+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b34157fb0 con 0x7f6b20003a60 2023-10-06T04:31:31.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.837+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b341507b0 con 0x7f6b20003a60 2023-10-06T04:31:31.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.838+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b3406c990 con 0x7f6b20003a60 2023-10-06T04:31:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.838+0000 7f6b2a7fc700 1 -- 172.21.15.123:0/1666193998 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6b24007590 con 0x7f6b20003a60 2023-10-06T04:31:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.839+0000 7f6b2a7fc700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6b14066f30 0x7f6b140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.839+0000 7f6b2a7fc700 1 -- 172.21.15.123:0/1666193998 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f6b24085130 con 0x7f6b20003a60 2023-10-06T04:31:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.839+0000 7f6b31d9b700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6b14066f30 0x7f6b140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.839+0000 7f6b31d9b700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6b14066f30 0x7f6b140693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:31.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:31.842+0000 7f6b2a7fc700 1 -- 172.21.15.123:0/1666193998 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6b24052610 con 0x7f6b20003a60 2023-10-06T04:31:32.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:32.027+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f6b340b2530 con 0x7f6b20003a60 2023-10-06T04:31:33.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.027+0000 7f6b2a7fc700 1 -- 172.21.15.123:0/1666193998 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v20) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6b240170b0 con 0x7f6b20003a60 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6b14066f30 msgr2=0x7f6b140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6b14066f30 0x7f6b140693e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f6b1c007fd0 tx=0x7f6b1c009040 comp rx=0 tx=0).stop 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 msgr2=0x7f6b341584d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b341584d0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f6b2000c0a0 tx=0x7f6b24015000 comp rx=0 tx=0).stop 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 shutdown_connections 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6b14066f30 0x7f6b140693e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6b20003a60 0x7f6b341584d0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6b34167a20 0x7f6b34167ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 --2- 172.21.15.123:0/1666193998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6b34156720 0x7f6b34156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 >> 172.21.15.123:0/1666193998 conn(0x7f6b340cd250 msgr2=0x7f6b34159950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:33.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 shutdown_connections 2023-10-06T04:31:33.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.030+0000 7f6b38ad2700 1 -- 172.21.15.123:0/1666193998 wait complete. 2023-10-06T04:31:33.033 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2023-10-06T04:31:33.047 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:31:33.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.177+0000 7ff02a7a0700 1 Processor -- start 2023-10-06T04:31:33.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.177+0000 7ff02a7a0700 1 -- start start 2023-10-06T04:31:33.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff02a7a0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff0240c8340 0x7ff0240a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:33.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff02a7a0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0240c8710 0x7ff0240c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:33.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff02a7a0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff024069aa0 con 0x7ff024068670 2023-10-06T04:31:33.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff02a7a0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff02406a340 con 0x7ff0240c8710 2023-10-06T04:31:33.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff02a7a0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0240aa6f0 con 0x7ff0240c8340 2023-10-06T04:31:33.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff0237fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff0240c8710 0x7ff0240c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff028d3d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff024068670 0x7ff0240c6d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33284/0 (socket says 172.21.15.123:33284) 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff028d3d700 1 -- 172.21.15.123:0/508276165 learned_addr learned my addr 172.21.15.123:0/508276165 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff0237fe700 1 --2- 172.21.15.123:0/508276165 >> v2:172.21.15.123:3300/0 conn(0x7ff0240c8710 0x7ff0240c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52140/0 (socket says 172.21.15.123:52140) 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff023fff700 1 --2- 172.21.15.123:0/508276165 >> v2:172.21.15.177:3300/0 conn(0x7ff0240c8340 0x7ff0240a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff0237fe700 1 -- 172.21.15.123:0/508276165 >> v2:172.21.15.177:3300/0 conn(0x7ff0240c8340 msgr2=0x7ff0240a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff0237fe700 1 --2- 172.21.15.123:0/508276165 >> v2:172.21.15.177:3300/0 conn(0x7ff0240c8340 0x7ff0240a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff0237fe700 1 -- 172.21.15.123:0/508276165 >> v1:172.21.15.123:6789/0 conn(0x7ff024068670 legacy=0x7ff0240c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff0237fe700 1 -- 172.21.15.123:0/508276165 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff024069cf0 con 0x7ff0240c8710 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.178+0000 7ff022ffd700 1 -- 172.21.15.123:0/508276165 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3671047661 0 0) 0x7ff024069aa0 con 0x7ff024068670 2023-10-06T04:31:33.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff0237fe700 1 --2- 172.21.15.123:0/508276165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff0240c8b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff02406d1c0 tx=0x7ff0140091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13d870257df3f159 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff022ffd700 1 -- 172.21.15.123:0/508276165 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01400b6b0 con 0x7ff0240c8710 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff022ffd700 1 -- 172.21.15.123:0/508276165 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01400b810 con 0x7ff0240c8710 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff022ffd700 1 -- 172.21.15.123:0/508276165 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01400bbb0 con 0x7ff0240c8710 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 -- 172.21.15.123:0/508276165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 msgr2=0x7ff0240c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/508276165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff0240c8b60 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7ff02406d1c0 tx=0x7ff0140091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 -- 172.21.15.123:0/508276165 shutdown_connections 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/508276165 >> v2:172.21.15.177:3300/0 conn(0x7ff0240c8340 0x7ff0240a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/508276165 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff0240c8b60 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 -- 172.21.15.123:0/508276165 >> 172.21.15.123:0/508276165 conn(0x7ff0240c9ca0 msgr2=0x7ff02406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:33.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 -- 172.21.15.123:0/508276165 shutdown_connections 2023-10-06T04:31:33.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.179+0000 7ff02a7a0700 1 -- 172.21.15.123:0/508276165 wait complete. 2023-10-06T04:31:33.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.180+0000 7ff02a7a0700 1 Processor -- start 2023-10-06T04:31:33.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff02a7a0700 1 -- start start 2023-10-06T04:31:33.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff02a7a0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff024068670 0x7ff024156950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:33.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff02a7a0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0240c8340 0x7ff024157870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:33.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff02a7a0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff024157db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff02a7a0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240abe40 con 0x7ff0240c8710 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff02a7a0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240ac020 con 0x7ff0240c8340 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff02a7a0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0240ac300 con 0x7ff024068670 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.181+0000 7ff028d3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff024157db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff024157db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52146/0 (socket says 172.21.15.123:52146) 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff0237fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0240c8340 0x7ff024157870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 -- 172.21.15.123:0/1376224049 learned_addr learned my addr 172.21.15.123:0/1376224049 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff023fff700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff024068670 0x7ff024156950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 -- 172.21.15.123:0/1376224049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff024068670 msgr2=0x7ff024156950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff024068670 0x7ff024156950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 -- 172.21.15.123:0/1376224049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0240c8340 msgr2=0x7ff024157870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0240c8340 0x7ff024157870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 -- 172.21.15.123:0/1376224049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff014009420 con 0x7ff0240c8710 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff028d3d700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff024157db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00cc90 tx=0x7ff01c001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff00bfff700 1 -- 172.21.15.123:0/1376224049 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c00a8b0 con 0x7ff0240c8710 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff00bfff700 1 -- 172.21.15.123:0/1376224049 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff01c00aa10 con 0x7ff0240c8710 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff00bfff700 1 -- 172.21.15.123:0/1376224049 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff01c00add0 con 0x7ff0240c8710 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0240ac5e0 con 0x7ff0240c8710 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.182+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff024156e90 con 0x7ff0240c8710 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.183+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0240ca0a0 con 0x7ff0240c8710 2023-10-06T04:31:33.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.183+0000 7ff00bfff700 1 -- 172.21.15.123:0/1376224049 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff01c007300 con 0x7ff0240c8710 2023-10-06T04:31:33.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.184+0000 7ff00bfff700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff010066f30 0x7ff0100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:33.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.184+0000 7ff00bfff700 1 -- 172.21.15.123:0/1376224049 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5576+0+0 (secure 0 0 0) 0x7ff01c0847d0 con 0x7ff0240c8710 2023-10-06T04:31:33.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.184+0000 7ff023fff700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff010066f30 0x7ff0100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:33.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.184+0000 7ff023fff700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff010066f30 0x7ff0100693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff018006fd0 tx=0x7ff018008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:33.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.188+0000 7ff00bfff700 1 -- 172.21.15.123:0/1376224049 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff01c051b40 con 0x7ff0240c8710 2023-10-06T04:31:33.342 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:33.344 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:31:33.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:33.372+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7ff024069fd0 con 0x7ff0240c8710 2023-10-06T04:31:34.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.029+0000 7ff00bfff700 1 -- 172.21.15.123:0/1376224049 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v21) v1 ==== 170+0+0 (secure 0 0 0) 0x7ff01c055160 con 0x7ff0240c8710 2023-10-06T04:31:34.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.037+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff010066f30 msgr2=0x7ff0100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:34.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.037+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff010066f30 0x7ff0100693e0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff018006fd0 tx=0x7ff018008040 comp rx=0 tx=0).stop 2023-10-06T04:31:34.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.037+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 msgr2=0x7ff024157db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:34.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.037+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff024157db0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff01c00cc90 tx=0x7ff01c001a80 comp rx=0 tx=0).stop 2023-10-06T04:31:34.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.038+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 shutdown_connections 2023-10-06T04:31:34.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.038+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff010066f30 0x7ff0100693e0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:34.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.038+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff0240c8710 0x7ff024157db0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:34.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.038+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff024068670 0x7ff024156950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:34.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.038+0000 7ff02a7a0700 1 --2- 172.21.15.123:0/1376224049 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff0240c8340 0x7ff024157870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:34.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.038+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 >> 172.21.15.123:0/1376224049 conn(0x7ff0240c9ca0 msgr2=0x7ff0240cacb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:34.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.039+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 shutdown_connections 2023-10-06T04:31:34.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.039+0000 7ff02a7a0700 1 -- 172.21.15.123:0/1376224049 wait complete. 2023-10-06T04:31:34.045 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2023-10-06T04:31:34.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:31:34.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.181+0000 7fabf4b3b700 1 Processor -- start 2023-10-06T04:31:34.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.181+0000 7fabf4b3b700 1 -- start start 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.182+0000 7fabf4b3b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fabf0068d80 0x7fabf00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.182+0000 7fabf4b3b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf0069aa0 con 0x7fabf00c8340 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.182+0000 7fabf4b3b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabf006a340 con 0x7fabf00c8710 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.182+0000 7fabf4b3b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabf00aa500 con 0x7fabf0068d80 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.182+0000 7fabeed9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fabf0068d80 0x7fabf00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.182+0000 7fabee59c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fabf00c8340 0x7fabf00a9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33286/0 (socket says 172.21.15.123:33286) 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.182+0000 7fabee59c700 1 -- 172.21.15.123:0/3276235323 learned_addr learned my addr 172.21.15.123:0/3276235323 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:34.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.183+0000 7fabeed9d700 1 -- 172.21.15.123:0/3276235323 >> v1:172.21.15.177:6789/0 conn(0x7fabf00c8710 legacy=0x7fabf0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.183+0000 7fabeed9d700 1 -- 172.21.15.123:0/3276235323 >> v1:172.21.15.123:6789/0 conn(0x7fabf00c8340 legacy=0x7fabf00a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.183+0000 7fabeed9d700 1 -- 172.21.15.123:0/3276235323 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabf00aa6f0 con 0x7fabf0068d80 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.183+0000 7fabeed9d700 1 --2- 172.21.15.123:0/3276235323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf0068d80 0x7fabf00c8b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009e60 tx=0x7fabe8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f448a523c99ac14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.183+0000 7fabed59a700 1 -- 172.21.15.123:0/3276235323 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe800e070 con 0x7fabf0068d80 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.183+0000 7fabed59a700 1 -- 172.21.15.123:0/3276235323 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe800bc30 con 0x7fabf0068d80 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.183+0000 7fabed59a700 1 -- 172.21.15.123:0/3276235323 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8012430 con 0x7fabf0068d80 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.184+0000 7fabf4b3b700 1 -- 172.21.15.123:0/3276235323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf0068d80 msgr2=0x7fabf00c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.184+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/3276235323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf0068d80 0x7fabf00c8b60 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fabe8009e60 tx=0x7fabe8009780 comp rx=0 tx=0).stop 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.184+0000 7fabf4b3b700 1 -- 172.21.15.123:0/3276235323 shutdown_connections 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.184+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/3276235323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf0068d80 0x7fabf00c8b60 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:34.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.184+0000 7fabf4b3b700 1 -- 172.21.15.123:0/3276235323 >> 172.21.15.123:0/3276235323 conn(0x7fabf00c9ca0 msgr2=0x7fabf006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:34.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.184+0000 7fabf4b3b700 1 -- 172.21.15.123:0/3276235323 shutdown_connections 2023-10-06T04:31:34.187 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.184+0000 7fabf4b3b700 1 -- 172.21.15.123:0/3276235323 wait complete. 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 Processor -- start 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 -- start start 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf0068d80 0x7fabf0156e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf00c8340 0x7fabf00c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf00c8710 0x7fabf0157c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf015b5c0 con 0x7fabf00c8710 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00c0bc0 con 0x7fabf00c8340 2023-10-06T04:31:34.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.185+0000 7fabf4b3b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabf00c0ea0 con 0x7fabf0068d80 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf00c8710 0x7fabf0157c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf00c8710 0x7fabf0157c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52162/0 (socket says 172.21.15.123:52162) 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 -- 172.21.15.123:0/1572985824 learned_addr learned my addr 172.21.15.123:0/1572985824 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabedd9b700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf00c8340 0x7fabf00c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabee59c700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf0068d80 0x7fabf0156e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 -- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf0068d80 msgr2=0x7fabf0156e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf0068d80 0x7fabf0156e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 -- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf00c8340 msgr2=0x7fabf00c0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf00c8340 0x7fabf00c0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 -- 172.21.15.123:0/1572985824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabe8009a20 con 0x7fabf00c8710 2023-10-06T04:31:34.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabeed9d700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf00c8710 0x7fabf0157c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabe80097c0 tx=0x7fabe8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:34.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabde7fc700 1 -- 172.21.15.123:0/1572985824 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe8012e10 con 0x7fabf00c8710 2023-10-06T04:31:34.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabde7fc700 1 -- 172.21.15.123:0/1572985824 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabe801e630 con 0x7fabf00c8710 2023-10-06T04:31:34.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabf00c1180 con 0x7fabf00c8710 2023-10-06T04:31:34.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabde7fc700 1 -- 172.21.15.123:0/1572985824 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabe801e9f0 con 0x7fabf00c8710 2023-10-06T04:31:34.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.186+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabf00c1740 con 0x7fabf00c8710 2023-10-06T04:31:34.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.187+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabf00ca0a0 con 0x7fabf00c8710 2023-10-06T04:31:34.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.188+0000 7fabde7fc700 1 -- 172.21.15.123:0/1572985824 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabe8024070 con 0x7fabf00c8710 2023-10-06T04:31:34.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.188+0000 7fabde7fc700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:34.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.188+0000 7fabde7fc700 1 -- 172.21.15.123:0/1572985824 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fabe8087340 con 0x7fabf00c8710 2023-10-06T04:31:34.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.188+0000 7fabee59c700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:34.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.189+0000 7fabee59c700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabd0066f30 0x7fabd00693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007130 tx=0x7fabe4007eb0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:34.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.192+0000 7fabde7fc700 1 -- 172.21.15.123:0/1572985824 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabe8054490 con 0x7fabf00c8710 2023-10-06T04:31:34.419 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:34.418+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fabf0069cf0 con 0x7fabf00c8710 2023-10-06T04:31:35.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.244+0000 7fabde7fc700 1 -- 172.21.15.123:0/1572985824 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 3 and data pool 4 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7fabe8057ab0 con 0x7fabf00c8710 2023-10-06T04:31:35.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabd0066f30 msgr2=0x7fabd00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabd0066f30 0x7fabd00693e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fabe4007130 tx=0x7fabe4007eb0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf00c8710 msgr2=0x7fabf0157c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf00c8710 0x7fabf0157c50 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fabe80097c0 tx=0x7fabe8019000 comp rx=0 tx=0).stop 2023-10-06T04:31:35.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 shutdown_connections 2023-10-06T04:31:35.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabd0066f30 0x7fabd00693e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabf00c8710 0x7fabf0157c50 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabf0068d80 0x7fabf0156e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 --2- 172.21.15.123:0/1572985824 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabf00c8340 0x7fabf00c0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 >> 172.21.15.123:0/1572985824 conn(0x7fabf00c9ca0 msgr2=0x7fabf0158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:35.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 shutdown_connections 2023-10-06T04:31:35.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.248+0000 7fabf4b3b700 1 -- 172.21.15.123:0/1572985824 wait complete. 2023-10-06T04:31:35.252 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 3 and data pool 4 2023-10-06T04:31:35.264 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:35.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.384+0000 7f51f35c6700 1 Processor -- start 2023-10-06T04:31:35.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.384+0000 7f51f35c6700 1 -- start start 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f35c6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51ec068d80 0x7f51ec0c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f35c6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f51ec0c7460 0x7f51ec06aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f35c6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51ec0c7c50 con 0x7f51ec155780 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f35c6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec0c7f30 con 0x7f51ec068d80 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f35c6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51ec0c8210 con 0x7f51ec0c7460 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51ec068d80 0x7f51ec0c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f51ec068d80 0x7f51ec0c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52168/0 (socket says 172.21.15.123:52168) 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 -- 172.21.15.123:0/4254391765 learned_addr learned my addr 172.21.15.123:0/4254391765 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f1b63700 1 --2- 172.21.15.123:0/4254391765 >> v2:172.21.15.177:3300/0 conn(0x7f51ec0c7460 0x7f51ec06aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 -- 172.21.15.123:0/4254391765 >> v2:172.21.15.177:3300/0 conn(0x7f51ec0c7460 msgr2=0x7f51ec06aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 --2- 172.21.15.123:0/4254391765 >> v2:172.21.15.177:3300/0 conn(0x7f51ec0c7460 0x7f51ec06aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 -- 172.21.15.123:0/4254391765 >> v1:172.21.15.177:6789/0 conn(0x7f51ec155780 legacy=0x7f51ec068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 -- 172.21.15.123:0/4254391765 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51ec0c84f0 con 0x7f51ec068d80 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f1b63700 1 --2- 172.21.15.123:0/4254391765 >> v2:172.21.15.177:3300/0 conn(0x7f51ec0c7460 0x7f51ec06aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51f0b61700 1 --2- 172.21.15.123:0/4254391765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec068d80 0x7f51ec0c6f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f51ec06d800 tx=0x7f51dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4e1fd61ee3cb6b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.385+0000 7f51e3fff700 1 -- 172.21.15.123:0/4254391765 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00b820 con 0x7f51ec068d80 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51e3fff700 1 -- 172.21.15.123:0/4254391765 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc00b980 con 0x7f51ec068d80 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51e3fff700 1 -- 172.21.15.123:0/4254391765 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc00bd20 con 0x7f51ec068d80 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 -- 172.21.15.123:0/4254391765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec068d80 msgr2=0x7f51ec0c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 --2- 172.21.15.123:0/4254391765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec068d80 0x7f51ec0c6f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f51ec06d800 tx=0x7f51dc009250 comp rx=0 tx=0).stop 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 -- 172.21.15.123:0/4254391765 shutdown_connections 2023-10-06T04:31:35.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 --2- 172.21.15.123:0/4254391765 >> v2:172.21.15.177:3300/0 conn(0x7f51ec0c7460 0x7f51ec06aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 --2- 172.21.15.123:0/4254391765 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec068d80 0x7f51ec0c6f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 -- 172.21.15.123:0/4254391765 >> 172.21.15.123:0/4254391765 conn(0x7f51ec0c9c90 msgr2=0x7f51ec06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:35.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 -- 172.21.15.123:0/4254391765 shutdown_connections 2023-10-06T04:31:35.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.386+0000 7f51f35c6700 1 -- 172.21.15.123:0/4254391765 wait complete. 2023-10-06T04:31:35.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.387+0000 7f51f35c6700 1 Processor -- start 2023-10-06T04:31:35.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.387+0000 7f51f35c6700 1 -- start start 2023-10-06T04:31:35.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f35c6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec068d80 0x7f51ec0ac200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f35c6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0c7460 0x7f51ec148e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f35c6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51ec155780 0x7f51ec0c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f35c6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec169c40 con 0x7f51ec0c7460 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f35c6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0af030 con 0x7f51ec068d80 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f35c6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51ec0ac740 con 0x7f51ec155780 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f0b61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0c7460 0x7f51ec148e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f0b61700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0c7460 0x7f51ec148e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52172/0 (socket says 172.21.15.123:52172) 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f0b61700 1 -- 172.21.15.123:0/2297603762 learned_addr learned my addr 172.21.15.123:0/2297603762 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f1b63700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51ec155780 0x7f51ec0c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.388+0000 7f51f1362700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec068d80 0x7f51ec0ac200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f0b61700 1 -- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51ec155780 msgr2=0x7f51ec0c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f0b61700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51ec155780 0x7f51ec0c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f0b61700 1 -- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec068d80 msgr2=0x7f51ec0ac200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f0b61700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec068d80 0x7f51ec0ac200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f0b61700 1 -- 172.21.15.123:0/2297603762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51dc0094f0 con 0x7f51ec0c7460 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f0b61700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0c7460 0x7f51ec148e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0030c0 tx=0x7f51dc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51e17fa700 1 -- 172.21.15.123:0/2297603762 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc011e60 con 0x7f51ec0c7460 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51e17fa700 1 -- 172.21.15.123:0/2297603762 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51dc020070 con 0x7f51ec0c7460 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51ec149360 con 0x7f51ec0c7460 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51e17fa700 1 -- 172.21.15.123:0/2297603762 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51dc01b6a0 con 0x7f51ec0c7460 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.389+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51ec0acc10 con 0x7f51ec0c7460 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.390+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51ec06cc70 con 0x7f51ec0c7460 2023-10-06T04:31:35.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.390+0000 7f51e17fa700 1 -- 172.21.15.123:0/2297603762 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f51dc022030 con 0x7f51ec0c7460 2023-10-06T04:31:35.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.390+0000 7f51e17fa700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f51d0066f30 0x7f51d00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.391+0000 7f51e17fa700 1 -- 172.21.15.123:0/2297603762 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f51dc0860e0 con 0x7f51ec0c7460 2023-10-06T04:31:35.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.391+0000 7f51f1362700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f51d0066f30 0x7f51d00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.391+0000 7f51f1362700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f51d0066f30 0x7f51d00693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:35.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.395+0000 7f51e17fa700 1 -- 172.21.15.123:0/2297603762 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f51dc0531f0 con 0x7f51ec0c7460 2023-10-06T04:31:35.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.579+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f51ec149360 con 0x7f51ec0c7460 2023-10-06T04:31:35.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.580+0000 7f51e17fa700 1 -- 172.21.15.123:0/2297603762 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17023 (secure 0 0 0) 0x7f51ec149360 con 0x7f51ec0c7460 2023-10-06T04:31:35.582 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:35.582 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:34.418878+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f51d0066f30 msgr2=0x7f51d00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f51d0066f30 0x7f51d00693e0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f51e8006fd0 tx=0x7f51e8008040 comp rx=0 tx=0).stop 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0c7460 msgr2=0x7f51ec148e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0c7460 0x7f51ec148e20 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f51dc0030c0 tx=0x7f51dc016000 comp rx=0 tx=0).stop 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 shutdown_connections 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f51d0066f30 0x7f51d00693e0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f51ec155780 0x7f51ec0c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f51ec068d80 0x7f51ec0ac200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 --2- 172.21.15.123:0/2297603762 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f51ec0c7460 0x7f51ec148e20 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 >> 172.21.15.123:0/2297603762 conn(0x7f51ec0c9c90 msgr2=0x7f51ec1541e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:35.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 shutdown_connections 2023-10-06T04:31:35.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.582+0000 7f51f35c6700 1 -- 172.21.15.123:0/2297603762 wait complete. 2023-10-06T04:31:35.596 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:35.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.711+0000 7f81dbc9e700 1 Processor -- start 2023-10-06T04:31:35.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.711+0000 7f81dbc9e700 1 -- start start 2023-10-06T04:31:35.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81dbc9e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f81d4067660 0x7f81d4067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81dbc9e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f81d40c9570 0x7f81d406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81dbc9e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81d4067f70 con 0x7f81d40ca5b0 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81dbc9e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d406a750 con 0x7f81d40c9570 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81dbc9e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81d406aa30 con 0x7f81d4067660 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d9239700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f81d40c9570 0x7f81d406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81da23b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f81d40ca5b0 0x7f81d406e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33288/0 (socket says 172.21.15.123:33288) 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81da23b700 1 -- 172.21.15.123:0/1916356693 learned_addr learned my addr 172.21.15.123:0/1916356693 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d9239700 1 -- 172.21.15.123:0/1916356693 >> v2:172.21.15.177:3300/0 conn(0x7f81d4067660 msgr2=0x7f81d4067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d9239700 1 --2- 172.21.15.123:0/1916356693 >> v2:172.21.15.177:3300/0 conn(0x7f81d4067660 0x7f81d4067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d9239700 1 -- 172.21.15.123:0/1916356693 >> v1:172.21.15.123:6789/0 conn(0x7f81d40ca5b0 legacy=0x7f81d406e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d9239700 1 -- 172.21.15.123:0/1916356693 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d406ad10 con 0x7f81d40c9570 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d8a38700 1 -- 172.21.15.123:0/1916356693 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3155280008 0 0) 0x7f81d4067f70 con 0x7f81d40ca5b0 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d9239700 1 --2- 172.21.15.123:0/1916356693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40c9570 0x7f81d406e250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81d4064ee0 tx=0x7f81c40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7ce7d5dac14ca0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d8a38700 1 -- 172.21.15.123:0/1916356693 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c400b6b0 con 0x7f81d40c9570 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.712+0000 7f81d8a38700 1 -- 172.21.15.123:0/1916356693 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c400b810 con 0x7f81d40c9570 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81d8a38700 1 -- 172.21.15.123:0/1916356693 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c400bbb0 con 0x7f81d40c9570 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 -- 172.21.15.123:0/1916356693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40c9570 msgr2=0x7f81d406e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/1916356693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40c9570 0x7f81d406e250 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f81d4064ee0 tx=0x7f81c40091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 -- 172.21.15.123:0/1916356693 shutdown_connections 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/1916356693 >> v2:172.21.15.177:3300/0 conn(0x7f81d4067660 0x7f81d4067a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/1916356693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40c9570 0x7f81d406e250 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 -- 172.21.15.123:0/1916356693 >> 172.21.15.123:0/1916356693 conn(0x7f81d40cda80 msgr2=0x7f81d40c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:35.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 -- 172.21.15.123:0/1916356693 shutdown_connections 2023-10-06T04:31:35.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.713+0000 7f81dbc9e700 1 -- 172.21.15.123:0/1916356693 wait complete. 2023-10-06T04:31:35.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.714+0000 7f81dbc9e700 1 Processor -- start 2023-10-06T04:31:35.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.714+0000 7f81dbc9e700 1 -- start start 2023-10-06T04:31:35.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81dbc9e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d4067660 0x7f81d4156630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81dbc9e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d40c9570 0x7f81d40b1b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81dbc9e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40ca5b0 0x7f81d40ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81dbc9e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d415b3a0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81dbc9e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81d416a800 con 0x7f81d4067660 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81dbc9e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81d4156b70 con 0x7f81d40c9570 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40ca5b0 0x7f81d40ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40ca5b0 0x7f81d40ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52200/0 (socket says 172.21.15.123:52200) 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 -- 172.21.15.123:0/4173983935 learned_addr learned my addr 172.21.15.123:0/4173983935 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81d9a3a700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d4067660 0x7f81d4156630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 -- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d40c9570 msgr2=0x7f81d40b1b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d40c9570 0x7f81d40b1b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 -- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d4067660 msgr2=0x7f81d4156630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d4067660 0x7f81d4156630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 -- 172.21.15.123:0/4173983935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c4009420 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81d9a3a700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d4067660 0x7f81d4156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.715+0000 7f81da23b700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40ca5b0 0x7f81d40ac8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00cc20 tx=0x7f81cc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.716+0000 7f81c9ffb700 1 -- 172.21.15.123:0/4173983935 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc004bb0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.716+0000 7f81c9ffb700 1 -- 172.21.15.123:0/4173983935 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81cc004d10 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.716+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81d416a520 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.716+0000 7f81c9ffb700 1 -- 172.21.15.123:0/4173983935 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81cc00abd0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.716+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81d40b1360 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.717+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81d40b0ee0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.717+0000 7f81c9ffb700 1 -- 172.21.15.123:0/4173983935 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f81cc0070f0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.717+0000 7f81c9ffb700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f81b8066f30 0x7f81b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.718+0000 7f81c9ffb700 1 -- 172.21.15.123:0/4173983935 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f81cc084480 con 0x7f81d40ca5b0 2023-10-06T04:31:35.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.718+0000 7f81d9a3a700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f81b8066f30 0x7f81b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:35.722 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.718+0000 7f81d9a3a700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f81b8066f30 0x7f81b80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81d00079b0 tx=0x7f81d0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:35.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.722+0000 7f81c9ffb700 1 -- 172.21.15.123:0/4173983935 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f81d40b0ee0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.905+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81d40b0ee0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.906+0000 7f81c9ffb700 1 -- 172.21.15.123:0/4173983935 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+17023 (secure 0 0 0) 0x7f81d40b0ee0 con 0x7f81d40ca5b0 2023-10-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:35.907 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:34.418878+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:35.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f81b8066f30 msgr2=0x7f81b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f81b8066f30 0x7f81b80693e0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f81d00079b0 tx=0x7f81d0008040 comp rx=0 tx=0).stop 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40ca5b0 msgr2=0x7f81d40ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40ca5b0 0x7f81d40ac8c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f81cc00cc20 tx=0x7f81cc0019b0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 shutdown_connections 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f81b8066f30 0x7f81b80693e0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f81d40ca5b0 0x7f81d40ac8c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f81d4067660 0x7f81d4156630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 --2- 172.21.15.123:0/4173983935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f81d40c9570 0x7f81d40b1b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 >> 172.21.15.123:0/4173983935 conn(0x7f81d40cda80 msgr2=0x7f81d40a9940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 shutdown_connections 2023-10-06T04:31:35.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:35.908+0000 7f81dbc9e700 1 -- 172.21.15.123:0/4173983935 wait complete. 2023-10-06T04:31:35.923 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:31:36.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.042+0000 7f8cdbd10700 1 Processor -- start 2023-10-06T04:31:36.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.042+0000 7f8cdbd10700 1 -- start start 2023-10-06T04:31:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cdbd10700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:36.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cdbd10700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8cd4068d80 0x7f8cd40c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cdbd10700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cd4069aa0 con 0x7f8cd40c8710 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cdbd10700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cd406a340 con 0x7f8cd4068d80 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cdbd10700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40aa6f0 con 0x7f8cd40c8340 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cd9aac700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cda2ad700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8cd4068d80 0x7f8cd40c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cd92ab700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f8cd40c8710 0x7f8cd4068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:51204/0 (socket says 172.21.15.123:51204) 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cda2ad700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8cd4068d80 0x7f8cd40c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:49710/0 (socket says 172.21.15.123:49710) 2023-10-06T04:31:36.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cd9aac700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:33734/0 (socket says 172.21.15.123:33734) 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.043+0000 7f8cd92ab700 1 -- 172.21.15.123:0/2543593665 learned_addr learned my addr 172.21.15.123:0/2543593665 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cda2ad700 1 -- 172.21.15.123:0/2543593665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 msgr2=0x7f8cd40a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cda2ad700 1 --2- 172.21.15.123:0/2543593665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cda2ad700 1 -- 172.21.15.123:0/2543593665 >> v1:172.21.15.177:6789/0 conn(0x7f8cd40c8710 legacy=0x7f8cd4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cda2ad700 1 -- 172.21.15.123:0/2543593665 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cd4069cf0 con 0x7f8cd4068d80 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cd8aaa700 1 -- 172.21.15.123:0/2543593665 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2889578 0 0) 0x7f8cd4069aa0 con 0x7f8cd40c8710 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cda2ad700 1 --2- 172.21.15.123:0/2543593665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 0x7f8cd40c8b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009e60 tx=0x7f8ccc009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=177472d3de6c06fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cd8aaa700 1 -- 172.21.15.123:0/2543593665 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc00e070 con 0x7f8cd4068d80 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cd8aaa700 1 -- 172.21.15.123:0/2543593665 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc00bb60 con 0x7f8cd4068d80 2023-10-06T04:31:36.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.044+0000 7f8cd8aaa700 1 -- 172.21.15.123:0/2543593665 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc012430 con 0x7f8cd4068d80 2023-10-06T04:31:36.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 -- 172.21.15.123:0/2543593665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 msgr2=0x7f8cd40c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/2543593665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 0x7f8cd40c8b60 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8ccc009e60 tx=0x7f8ccc009780 comp rx=0 tx=0).stop 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 -- 172.21.15.123:0/2543593665 shutdown_connections 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/2543593665 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 0x7f8cd40c8b60 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/2543593665 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 -- 172.21.15.123:0/2543593665 >> 172.21.15.123:0/2543593665 conn(0x7f8cd40c9ca0 msgr2=0x7f8cd406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 -- 172.21.15.123:0/2543593665 shutdown_connections 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.045+0000 7f8cdbd10700 1 -- 172.21.15.123:0/2543593665 wait complete. 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.046+0000 7f8cdbd10700 1 Processor -- start 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.046+0000 7f8cdbd10700 1 -- start start 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cdbd10700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 0x7f8cd40ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cdbd10700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40ad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:36.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cdbd10700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd40c8710 0x7f8cd40c1a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cdbd10700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd41589b0 con 0x7f8cd40c8710 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cdbd10700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40ad010 con 0x7f8cd4068d80 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cdbd10700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cd40acb80 con 0x7f8cd40c8340 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd40c8710 0x7f8cd40c1a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd40c8710 0x7f8cd40c1a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52210/0 (socket says 172.21.15.123:52210) 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 -- 172.21.15.123:0/3143726805 learned_addr learned my addr 172.21.15.123:0/3143726805 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cd9aac700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 0x7f8cd40ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cd92ab700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40ad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 -- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 msgr2=0x7f8cd40ad600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40ad600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:36.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 -- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 msgr2=0x7f8cd40ae6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 0x7f8cd40ae6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 -- 172.21.15.123:0/3143726805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ccc009a20 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cda2ad700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd40c8710 0x7f8cd40c1a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8cd406d190 tx=0x7f8ccc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cc9ffb700 1 -- 172.21.15.123:0/3143726805 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc027470 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cc9ffb700 1 -- 172.21.15.123:0/3143726805 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ccc0275d0 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cc9ffb700 1 -- 172.21.15.123:0/3143726805 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ccc027990 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.047+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cd40aec20 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.048+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cd40ac250 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.048+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cd40ca0a0 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.049+0000 7f8cc9ffb700 1 -- 172.21.15.123:0/3143726805 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ccc024070 con 0x7f8cd40c8710 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.049+0000 7f8cc9ffb700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8cb8066f30 0x7f8cb80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:36.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.049+0000 7f8cc9ffb700 1 -- 172.21.15.123:0/3143726805 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ccc086cd0 con 0x7f8cd40c8710 2023-10-06T04:31:36.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.049+0000 7f8cd9aac700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8cb8066f30 0x7f8cb80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:36.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.050+0000 7f8cd9aac700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8cb8066f30 0x7f8cb80693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8cd416cf70 tx=0x7f8cd00091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:36.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.054+0000 7f8cc9ffb700 1 -- 172.21.15.123:0/3143726805 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ccc053de0 con 0x7f8cd40c8710 2023-10-06T04:31:36.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:36.279+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f8cd4069fd0 con 0x7f8cd40c8710 2023-10-06T04:31:37.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.250+0000 7f8cc9ffb700 1 -- 172.21.15.123:0/3143726805 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f8ccc02c0b0 con 0x7f8cd40c8710 2023-10-06T04:31:37.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8cb8066f30 msgr2=0x7f8cb80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:37.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8cb8066f30 0x7f8cb80693e0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8cd416cf70 tx=0x7f8cd00091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd40c8710 msgr2=0x7f8cd40c1a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd40c8710 0x7f8cd40c1a40 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8cd406d190 tx=0x7f8ccc019000 comp rx=0 tx=0).stop 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 shutdown_connections 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8cb8066f30 0x7f8cb80693e0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8cd40c8710 0x7f8cd40c1a40 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8cd4068d80 0x7f8cd40ae6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 --2- 172.21.15.123:0/3143726805 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8cd40c8340 0x7f8cd40ad600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 >> 172.21.15.123:0/3143726805 conn(0x7f8cd40c9ca0 msgr2=0x7f8cd40cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 shutdown_connections 2023-10-06T04:31:37.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.253+0000 7f8cdbd10700 1 -- 172.21.15.123:0/3143726805 wait complete. 2023-10-06T04:31:37.269 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:31:37.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.393+0000 7ffbc93a1700 1 Processor -- start 2023-10-06T04:31:37.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.393+0000 7ffbc93a1700 1 -- start start 2023-10-06T04:31:37.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc93a1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbc415a4d0 0x7ffbc406d950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:37.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc93a1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ffbc40c8ba0 0x7ffbc40abe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:37.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc93a1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbc4068c40 con 0x7ffbc406f900 2023-10-06T04:31:37.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc93a1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc4068310 con 0x7ffbc415a4d0 2023-10-06T04:31:37.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc93a1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbc40685f0 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbc415a4d0 0x7ffbc406d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ffbc415a4d0 0x7ffbc406d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52214/0 (socket says 172.21.15.123:52214) 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.394+0000 7ffbc3fff700 1 -- 172.21.15.123:0/3446884822 learned_addr learned my addr 172.21.15.123:0/3446884822 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc8ba0700 1 --2- 172.21.15.123:0/3446884822 >> v2:172.21.15.177:3300/0 conn(0x7ffbc40c8ba0 0x7ffbc40abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc3fff700 1 -- 172.21.15.123:0/3446884822 >> v2:172.21.15.177:3300/0 conn(0x7ffbc40c8ba0 msgr2=0x7ffbc40abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc3fff700 1 --2- 172.21.15.123:0/3446884822 >> v2:172.21.15.177:3300/0 conn(0x7ffbc40c8ba0 0x7ffbc40abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc3fff700 1 -- 172.21.15.123:0/3446884822 >> v1:172.21.15.177:6789/0 conn(0x7ffbc406f900 legacy=0x7ffbc40c8490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc3fff700 1 -- 172.21.15.123:0/3446884822 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbc40688d0 con 0x7ffbc415a4d0 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc3fff700 1 --2- 172.21.15.123:0/3446884822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc415a4d0 0x7ffbc406d950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffbc406c4e0 tx=0x7ffbac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57477404cb077892 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc2ffd700 1 -- 172.21.15.123:0/3446884822 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00b860 con 0x7ffbc415a4d0 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc2ffd700 1 -- 172.21.15.123:0/3446884822 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbac00b9c0 con 0x7ffbc415a4d0 2023-10-06T04:31:37.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc2ffd700 1 -- 172.21.15.123:0/3446884822 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbac00bd80 con 0x7ffbc415a4d0 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc93a1700 1 -- 172.21.15.123:0/3446884822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc415a4d0 msgr2=0x7ffbc406d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.395+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/3446884822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc415a4d0 0x7ffbc406d950 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7ffbc406c4e0 tx=0x7ffbac009250 comp rx=0 tx=0).stop 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.396+0000 7ffbc93a1700 1 -- 172.21.15.123:0/3446884822 shutdown_connections 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.396+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/3446884822 >> v2:172.21.15.177:3300/0 conn(0x7ffbc40c8ba0 0x7ffbc40abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.396+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/3446884822 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc415a4d0 0x7ffbc406d950 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.396+0000 7ffbc93a1700 1 -- 172.21.15.123:0/3446884822 >> 172.21.15.123:0/3446884822 conn(0x7ffbc40c90a0 msgr2=0x7ffbc40c94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.396+0000 7ffbc93a1700 1 -- 172.21.15.123:0/3446884822 shutdown_connections 2023-10-06T04:31:37.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.396+0000 7ffbc93a1700 1 -- 172.21.15.123:0/3446884822 wait complete. 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.397+0000 7ffbc93a1700 1 Processor -- start 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.397+0000 7ffbc93a1700 1 -- start start 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbc406f900 0x7ffbc40aabe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc40c8ba0 0x7ffbc415adc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc415a4d0 0x7ffbc415b300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc4168980 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40c5280 con 0x7ffbc415a4d0 2023-10-06T04:31:37.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbc40b0020 con 0x7ffbc406f900 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc40c8ba0 0x7ffbc415adc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc40c8ba0 0x7ffbc415adc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52226/0 (socket says 172.21.15.123:52226) 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 -- 172.21.15.123:0/4263332952 learned_addr learned my addr 172.21.15.123:0/4263332952 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc8ba0700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc415a4d0 0x7ffbc415b300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc3fff700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbc406f900 0x7ffbc40aabe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 -- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbc406f900 msgr2=0x7ffbc40aabe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbc406f900 0x7ffbc40aabe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 -- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc415a4d0 msgr2=0x7ffbc415b300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc415a4d0 0x7ffbc415b300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 -- 172.21.15.123:0/4263332952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbac0094f0 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc37fe700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc40c8ba0 0x7ffbc415adc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c590 tx=0x7ffbb400c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:37.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbbbfff700 1 -- 172.21.15.123:0/4263332952 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4011590 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbbbfff700 1 -- 172.21.15.123:0/4263332952 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbb4011710 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbc4168f30 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbbbfff700 1 -- 172.21.15.123:0/4263332952 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbb4011ad0 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.398+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbc41686a0 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.399+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbc4064b00 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.400+0000 7ffbbbfff700 1 -- 172.21.15.123:0/4263332952 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ffbb401b3b0 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.400+0000 7ffbbbfff700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.400+0000 7ffbc3fff700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.400+0000 7ffbbbfff700 1 -- 172.21.15.123:0/4263332952 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ffbb4083a60 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.400+0000 7ffbc3fff700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffbc406c4e0 tx=0x7ffbac017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:37.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.404+0000 7ffbbbfff700 1 -- 172.21.15.123:0/4263332952 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ffbb4050b70 con 0x7ffbc40c8ba0 2023-10-06T04:31:37.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:37.629+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ffbc40c53c0 con 0x7ffbc40c8ba0 2023-10-06T04:31:38.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.257+0000 7ffbbbfff700 1 -- 172.21.15.123:0/4263332952 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7ffbb4017070 con 0x7ffbc40c8ba0 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ffba4066f30 msgr2=0x7ffba40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ffba4066f30 0x7ffba40693e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7ffbc406c4e0 tx=0x7ffbac017000 comp rx=0 tx=0).stop 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc40c8ba0 msgr2=0x7ffbc415adc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc40c8ba0 0x7ffbc415adc0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7ffbb400c590 tx=0x7ffbb400c8a0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 shutdown_connections 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ffba4066f30 0x7ffba40693e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ffbc415a4d0 0x7ffbc415b300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ffbc406f900 0x7ffbc40aabe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 --2- 172.21.15.123:0/4263332952 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ffbc40c8ba0 0x7ffbc415adc0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 >> 172.21.15.123:0/4263332952 conn(0x7ffbc40c90a0 msgr2=0x7ffbc4159550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:38.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 shutdown_connections 2023-10-06T04:31:38.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.259+0000 7ffbc93a1700 1 -- 172.21.15.123:0/4263332952 wait complete. 2023-10-06T04:31:38.273 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:31:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.391+0000 7f3d05b3f700 1 Processor -- start 2023-10-06T04:31:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.391+0000 7f3d05b3f700 1 -- start start 2023-10-06T04:31:38.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3d05b3f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d000c9570 0x7f3d000a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3d05b3f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3d000ca5b0 0x7f3d000a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3d05b3f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d0006a750 con 0x7f3d00067660 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3d05b3f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0006aa30 con 0x7f3d000c9570 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3d05b3f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d0006ad10 con 0x7f3d000ca5b0 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3cfeffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d000c9570 0x7f3d000a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3cfeffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d000c9570 0x7f3d000a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52230/0 (socket says 172.21.15.123:52230) 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.392+0000 7f3cfeffd700 1 -- 172.21.15.123:0/2725981718 learned_addr learned my addr 172.21.15.123:0/2725981718 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfffff700 1 --2- 172.21.15.123:0/2725981718 >> v2:172.21.15.177:3300/0 conn(0x7f3d000ca5b0 0x7f3d000a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfeffd700 1 -- 172.21.15.123:0/2725981718 >> v2:172.21.15.177:3300/0 conn(0x7f3d000ca5b0 msgr2=0x7f3d000a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfeffd700 1 --2- 172.21.15.123:0/2725981718 >> v2:172.21.15.177:3300/0 conn(0x7f3d000ca5b0 0x7f3d000a9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfeffd700 1 -- 172.21.15.123:0/2725981718 >> v1:172.21.15.177:6789/0 conn(0x7f3d00067660 legacy=0x7f3d00067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfeffd700 1 -- 172.21.15.123:0/2725981718 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d0006aff0 con 0x7f3d000c9570 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfeffd700 1 --2- 172.21.15.123:0/2725981718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d000c9570 0x7f3d000a9940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d0006e150 tx=0x7f3ce8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e02b63550062eda6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfe7fc700 1 -- 172.21.15.123:0/2725981718 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800b820 con 0x7f3d000c9570 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfe7fc700 1 -- 172.21.15.123:0/2725981718 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ce800b980 con 0x7f3d000c9570 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3cfe7fc700 1 -- 172.21.15.123:0/2725981718 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ce800bd20 con 0x7f3d000c9570 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3d05b3f700 1 -- 172.21.15.123:0/2725981718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d000c9570 msgr2=0x7f3d000a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/2725981718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d000c9570 0x7f3d000a9940 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f3d0006e150 tx=0x7f3ce8009250 comp rx=0 tx=0).stop 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3d05b3f700 1 -- 172.21.15.123:0/2725981718 shutdown_connections 2023-10-06T04:31:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/2725981718 >> v2:172.21.15.177:3300/0 conn(0x7f3d000ca5b0 0x7f3d000a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/2725981718 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d000c9570 0x7f3d000a9940 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.393+0000 7f3d05b3f700 1 -- 172.21.15.123:0/2725981718 >> 172.21.15.123:0/2725981718 conn(0x7f3d000cda80 msgr2=0x7f3d000c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.394+0000 7f3d05b3f700 1 -- 172.21.15.123:0/2725981718 shutdown_connections 2023-10-06T04:31:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.394+0000 7f3d05b3f700 1 -- 172.21.15.123:0/2725981718 wait complete. 2023-10-06T04:31:38.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.395+0000 7f3d05b3f700 1 Processor -- start 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.395+0000 7f3d05b3f700 1 -- start start 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.395+0000 7f3d05b3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d00067660 0x7f3d000b11a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3d05b3f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000c9570 0x7f3d000b16e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3d05b3f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000ca5b0 0x7f3d000b1c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3d05b3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000b2160 con 0x7f3d00067660 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3d05b3f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000c3580 con 0x7f3d000ca5b0 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3d05b3f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d000aca80 con 0x7f3d000c9570 2023-10-06T04:31:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d00067660 0x7f3d000b11a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d00067660 0x7f3d000b11a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52240/0 (socket says 172.21.15.123:52240) 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 -- 172.21.15.123:0/1752238354 learned_addr learned my addr 172.21.15.123:0/1752238354 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cfeffd700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000c9570 0x7f3d000b16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cfffff700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000ca5b0 0x7f3d000b1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 -- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000c9570 msgr2=0x7f3d000b16e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000c9570 0x7f3d000b16e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 -- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000ca5b0 msgr2=0x7f3d000b1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000ca5b0 0x7f3d000b1c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 -- 172.21.15.123:0/1752238354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ce80094f0 con 0x7f3d00067660 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cff7fe700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d00067660 0x7f3d000b11a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000b640 tx=0x7f3cf0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cf77fe700 1 -- 172.21.15.123:0/1752238354 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000f510 con 0x7f3d00067660 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cf77fe700 1 -- 172.21.15.123:0/1752238354 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cf000f690 con 0x7f3d00067660 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d000c2ce0 con 0x7f3d00067660 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3cf77fe700 1 -- 172.21.15.123:0/1752238354 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cf000fa50 con 0x7f3d00067660 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.396+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d000c32a0 con 0x7f3d00067660 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.397+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3d000cde80 con 0x7f3d00067660 2023-10-06T04:31:38.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.398+0000 7f3cf77fe700 1 -- 172.21.15.123:0/1752238354 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3cf001a400 con 0x7f3d00067660 2023-10-06T04:31:38.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.398+0000 7f3cf77fe700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.398+0000 7f3cf77fe700 1 -- 172.21.15.123:0/1752238354 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3cf0084100 con 0x7f3d00067660 2023-10-06T04:31:38.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.398+0000 7f3cfeffd700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.398+0000 7f3cfeffd700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3ce0066f30 0x7f3ce00693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d0006e150 tx=0x7f3ce8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:38.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.402+0000 7f3cf77fe700 1 -- 172.21.15.123:0/1752238354 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3cf0051290 con 0x7f3d00067660 2023-10-06T04:31:38.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.626+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3d000a98b0 con 0x7f3d00067660 2023-10-06T04:31:38.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.627+0000 7f3cf77fe700 1 -- 172.21.15.123:0/1752238354 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f3cf00548b0 con 0x7f3d00067660 2023-10-06T04:31:38.628 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:38.629 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:31:38.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3ce0066f30 msgr2=0x7f3ce00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3ce0066f30 0x7f3ce00693e0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f3d0006e150 tx=0x7f3ce8016000 comp rx=0 tx=0).stop 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d00067660 msgr2=0x7f3d000b11a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d00067660 0x7f3d000b11a0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f3cf000b640 tx=0x7f3cf0009120 comp rx=0 tx=0).stop 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 shutdown_connections 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3ce0066f30 0x7f3ce00693e0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d00067660 0x7f3d000b11a0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d000ca5b0 0x7f3d000b1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 --2- 172.21.15.123:0/1752238354 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d000c9570 0x7f3d000b16e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 >> 172.21.15.123:0/1752238354 conn(0x7f3d000cda80 msgr2=0x7f3d00158380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 shutdown_connections 2023-10-06T04:31:38.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.629+0000 7f3d05b3f700 1 -- 172.21.15.123:0/1752238354 wait complete. 2023-10-06T04:31:38.632 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:31:38.642 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.758+0000 7ff7c1760700 1 Processor -- start 2023-10-06T04:31:38.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.758+0000 7ff7c1760700 1 -- start start 2023-10-06T04:31:38.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7c1760700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7bc06a540 0x7ff7bc15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7c1760700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7bc0690a0 con 0x7ff7bc06bb60 2023-10-06T04:31:38.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7c1760700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7bc15dd40 con 0x7ff7bc13f600 2023-10-06T04:31:38.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7c1760700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7bc068770 con 0x7ff7bc06a540 2023-10-06T04:31:38.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7ba7fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff7bc06bb60 0x7ff7bc15d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33302/0 (socket says 172.21.15.123:33302) 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7ba7fc700 1 -- 172.21.15.123:0/2360230771 learned_addr learned my addr 172.21.15.123:0/2360230771 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2197773531 0 0) 0x7ff7bc0690a0 con 0x7ff7bc06bb60 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff7a4003570 con 0x7ff7bc06bb60 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/2360230771 >> v2:172.21.15.177:3300/0 conn(0x7ff7bc06a540 0x7ff7bc15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3242950748 0 0) 0x7ff7a4003570 con 0x7ff7bc06bb60 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 >> v2:172.21.15.177:3300/0 conn(0x7ff7bc06a540 msgr2=0x7ff7bc15d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 --2- 172.21.15.123:0/2360230771 >> v2:172.21.15.177:3300/0 conn(0x7ff7bc06a540 0x7ff7bc15d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 >> v1:172.21.15.177:6789/0 conn(0x7ff7bc13f600 legacy=0x7ff7bc06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc068a50 con 0x7ff7bc06bb60 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7ff7ac001470 con 0x7ff7bc06bb60 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 >> v1:172.21.15.123:6789/0 conn(0x7ff7bc06bb60 legacy=0x7ff7bc15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a4003a60 0x7ff7a4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7a4004810 0x7ff7a4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7a400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc068770 con 0x7ff7a40052e0 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a400c190 con 0x7ff7a4003a60 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7a400c350 con 0x7ff7a4004810 2023-10-06T04:31:38.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.759+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/2360230771 >> v2:172.21.15.177:3300/0 conn(0x7ff7bc06a540 0x7ff7bc15d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7a400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7baffd700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a4003a60 0x7ff7a4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7ba7fc700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7a4004810 0x7ff7a4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 -- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7a4004810 msgr2=0x7ff7a4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7a4004810 0x7ff7a4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 -- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a4003a60 msgr2=0x7ff7a4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7a4003a60 0x7ff7a4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 -- 172.21.15.123:0/2360230771 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7a4003570 con 0x7ff7a40052e0 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7ba7fc700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7a4004810 0x7ff7a4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:38.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 -- 172.21.15.123:0/2360230771 reap_dead start 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7a400ba60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4007580 tx=0x7ff7b40097a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9efc0d70eff22c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4009ad0 con 0x7ff7a40052e0 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b4007dd0 con 0x7ff7a40052e0 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.760+0000 7ff7b9ffb700 1 -- 172.21.15.123:0/2360230771 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4014650 con 0x7ff7a40052e0 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.761+0000 7ff7c1760700 1 -- 172.21.15.123:0/2360230771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 msgr2=0x7ff7a400ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.761+0000 7ff7c1760700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7a400ba60 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff7b4007580 tx=0x7ff7b40097a0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.761+0000 7ff7c1760700 1 -- 172.21.15.123:0/2360230771 shutdown_connections 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.761+0000 7ff7c1760700 1 --2- 172.21.15.123:0/2360230771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7a400ba60 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.761+0000 7ff7c1760700 1 -- 172.21.15.123:0/2360230771 >> 172.21.15.123:0/2360230771 conn(0x7ff7bc0ce0c0 msgr2=0x7ff7bc0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:38.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.761+0000 7ff7c1760700 1 -- 172.21.15.123:0/2360230771 shutdown_connections 2023-10-06T04:31:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.761+0000 7ff7c1760700 1 -- 172.21.15.123:0/2360230771 wait complete. 2023-10-06T04:31:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.762+0000 7ff7c1760700 1 Processor -- start 2023-10-06T04:31:38.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.762+0000 7ff7c1760700 1 -- start start 2023-10-06T04:31:38.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7c1760700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7bc169770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7c1760700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7bc167a30 0x7ff7bc16a840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7c1760700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7bc0b29a0 0x7ff7bc0b2e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7c1760700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc16afc0 con 0x7ff7a40052e0 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7c1760700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc169cb0 con 0x7ff7bc167a30 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7c1760700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7bc0b3380 con 0x7ff7bc0b29a0 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7baffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7bc169770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7baffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7bc169770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52256/0 (socket says 172.21.15.123:52256) 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7baffd700 1 -- 172.21.15.123:0/4051978301 learned_addr learned my addr 172.21.15.123:0/4051978301 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:38.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7ba7fc700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7bc167a30 0x7ff7bc16a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.763+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7bc0b29a0 0x7ff7bc0b2e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7baffd700 1 -- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7bc0b29a0 msgr2=0x7ff7bc0b2e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7baffd700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7bc0b29a0 0x7ff7bc0b2e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7baffd700 1 -- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7bc167a30 msgr2=0x7ff7bc16a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7baffd700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7bc167a30 0x7ff7bc16a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7baffd700 1 -- 172.21.15.123:0/4051978301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7bc16ad80 con 0x7ff7a40052e0 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7ba7fc700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7bc167a30 0x7ff7bc16a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7bb7fe700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7bc0b29a0 0x7ff7bc0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7baffd700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7bc169770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0002980 tx=0x7ff7b0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:38.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7a2ffd700 1 -- 172.21.15.123:0/4051978301 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000f8d0 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7a2ffd700 1 -- 172.21.15.123:0/4051978301 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b000fa30 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7bc16b210 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7a2ffd700 1 -- 172.21.15.123:0/4051978301 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b000fdd0 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.764+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7bc16b7d0 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.765+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7bc0cd890 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.765+0000 7ff7a2ffd700 1 -- 172.21.15.123:0/4051978301 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7b0013600 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.766+0000 7ff7a2ffd700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7a8066f30 0x7ff7a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.766+0000 7ff7a2ffd700 1 -- 172.21.15.123:0/4051978301 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff7b0084480 con 0x7ff7a40052e0 2023-10-06T04:31:38.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.766+0000 7ff7ba7fc700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7a8066f30 0x7ff7a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:38.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.766+0000 7ff7ba7fc700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7a8066f30 0x7ff7a80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc158880 tx=0x7ff7ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:38.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.770+0000 7ff7a2ffd700 1 -- 172.21.15.123:0/4051978301 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7b0051590 con 0x7ff7a40052e0 2023-10-06T04:31:38.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.953+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7bc068d30 con 0x7ff7a40052e0 2023-10-06T04:31:38.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.954+0000 7ff7a2ffd700 1 -- 172.21.15.123:0/4051978301 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7ff7b0054bb0 con 0x7ff7a40052e0 2023-10-06T04:31:38.955 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:38.955 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:38.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.955+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7a8066f30 msgr2=0x7ff7a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.955+0000 7ff7c1760700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7a8066f30 0x7ff7a80693e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7ff7bc158880 tx=0x7ff7ac009040 comp rx=0 tx=0).stop 2023-10-06T04:31:38.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.955+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 msgr2=0x7ff7bc169770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:38.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.955+0000 7ff7c1760700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7bc169770 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7b0002980 tx=0x7ff7b0007d30 comp rx=0 tx=0).stop 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 shutdown_connections 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7a8066f30 0x7ff7a80693e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7a40052e0 0x7ff7bc169770 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7bc0b29a0 0x7ff7bc0b2e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 --2- 172.21.15.123:0/4051978301 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7bc167a30 0x7ff7bc16a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 >> 172.21.15.123:0/4051978301 conn(0x7ff7bc0ce0c0 msgr2=0x7ff7bc0cca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 shutdown_connections 2023-10-06T04:31:38.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:38.956+0000 7ff7c1760700 1 -- 172.21.15.123:0/4051978301 wait complete. 2023-10-06T04:31:38.969 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:31:38.969 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:31:39.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.085+0000 7f0ea3842700 1 Processor -- start 2023-10-06T04:31:39.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.085+0000 7f0ea3842700 1 -- start start 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea3842700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e9c1597e0 0x7f0e9c068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea3842700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0e9c068bb0 0x7f0e9c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea3842700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e9c06af00 con 0x7f0e9c159410 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea3842700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c0c8050 con 0x7f0e9c1597e0 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea3842700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e9c069010 con 0x7f0e9c068bb0 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea0ddd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e9c1597e0 0x7f0e9c068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea0ddd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e9c1597e0 0x7f0e9c068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52264/0 (socket says 172.21.15.123:52264) 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea0ddd700 1 -- 172.21.15.123:0/1557586490 learned_addr learned my addr 172.21.15.123:0/1557586490 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea0ddd700 1 -- 172.21.15.123:0/1557586490 >> v2:172.21.15.177:3300/0 conn(0x7f0e9c068bb0 msgr2=0x7f0e9c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:39.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea0ddd700 1 --2- 172.21.15.123:0/1557586490 >> v2:172.21.15.177:3300/0 conn(0x7f0e9c068bb0 0x7f0e9c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea0ddd700 1 -- 172.21.15.123:0/1557586490 >> v1:172.21.15.177:6789/0 conn(0x7f0e9c159410 legacy=0x7f0e9c0af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.086+0000 7f0ea0ddd700 1 -- 172.21.15.123:0/1557586490 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e9c06b460 con 0x7f0e9c1597e0 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0ea0ddd700 1 --2- 172.21.15.123:0/1557586490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c1597e0 0x7f0e9c068670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cc610 tx=0x7f0e98009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d898e8065af1d40b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0e93fff700 1 -- 172.21.15.123:0/1557586490 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800b820 con 0x7f0e9c1597e0 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0e93fff700 1 -- 172.21.15.123:0/1557586490 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e9800b980 con 0x7f0e9c1597e0 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0e93fff700 1 -- 172.21.15.123:0/1557586490 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e9800bd20 con 0x7f0e9c1597e0 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0ea3842700 1 -- 172.21.15.123:0/1557586490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c1597e0 msgr2=0x7f0e9c068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1557586490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c1597e0 0x7f0e9c068670 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cc610 tx=0x7f0e98009250 comp rx=0 tx=0).stop 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0ea3842700 1 -- 172.21.15.123:0/1557586490 shutdown_connections 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1557586490 >> v2:172.21.15.177:3300/0 conn(0x7f0e9c068bb0 0x7f0e9c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1557586490 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c1597e0 0x7f0e9c068670 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.087+0000 7f0ea3842700 1 -- 172.21.15.123:0/1557586490 >> 172.21.15.123:0/1557586490 conn(0x7f0e9c0c9c90 msgr2=0x7f0e9c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:39.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.088+0000 7f0ea3842700 1 -- 172.21.15.123:0/1557586490 shutdown_connections 2023-10-06T04:31:39.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.088+0000 7f0ea3842700 1 -- 172.21.15.123:0/1557586490 wait complete. 2023-10-06T04:31:39.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 Processor -- start 2023-10-06T04:31:39.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 -- start start 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c068bb0 0x7f0e9c1542b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e9c159410 0x7f0e9c13e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9c1597e0 0x7f0e9c156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c13fa20 con 0x7f0e9c068bb0 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c13ee10 con 0x7f0e9c1597e0 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.089+0000 7f0ea3842700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e9c13f0f0 con 0x7f0e9c159410 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c068bb0 0x7f0e9c1542b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c068bb0 0x7f0e9c1542b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52274/0 (socket says 172.21.15.123:52274) 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 -- 172.21.15.123:0/1814740268 learned_addr learned my addr 172.21.15.123:0/1814740268 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea0ddd700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e9c159410 0x7f0e9c13e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea1ddf700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9c1597e0 0x7f0e9c156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 -- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e9c159410 msgr2=0x7f0e9c13e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e9c159410 0x7f0e9c13e5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 -- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9c1597e0 msgr2=0x7f0e9c156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9c1597e0 0x7f0e9c156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 -- 172.21.15.123:0/1814740268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e980094f0 con 0x7f0e9c068bb0 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea0ddd700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e9c159410 0x7f0e9c13e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea15de700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c068bb0 0x7f0e9c1542b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b600 tx=0x7f0e8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0e917fa700 1 -- 172.21.15.123:0/1814740268 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00f510 con 0x7f0e9c068bb0 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0e917fa700 1 -- 172.21.15.123:0/1814740268 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e8c00f670 con 0x7f0e9c068bb0 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e9c13f3d0 con 0x7f0e9c068bb0 2023-10-06T04:31:39.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0e917fa700 1 -- 172.21.15.123:0/1814740268 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e8c00fa30 con 0x7f0e9c068bb0 2023-10-06T04:31:39.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.090+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e9c13eb20 con 0x7f0e9c068bb0 2023-10-06T04:31:39.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.091+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e9c06d6a0 con 0x7f0e9c068bb0 2023-10-06T04:31:39.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.091+0000 7f0e917fa700 1 -- 172.21.15.123:0/1814740268 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e8c01a400 con 0x7f0e9c068bb0 2023-10-06T04:31:39.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.092+0000 7f0e917fa700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e80066f30 0x7f0e800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.092+0000 7f0e917fa700 1 -- 172.21.15.123:0/1814740268 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e8c084160 con 0x7f0e9c068bb0 2023-10-06T04:31:39.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.092+0000 7f0ea0ddd700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e80066f30 0x7f0e800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.092+0000 7f0ea0ddd700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e80066f30 0x7f0e800693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cc610 tx=0x7f0e98016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.096+0000 7f0e917fa700 1 -- 172.21.15.123:0/1814740268 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e8c0512f0 con 0x7f0e9c068bb0 2023-10-06T04:31:39.132 INFO:tasks.ceph.mds.c.smithi123.stderr:2023-10-06T04:31:39.130+0000 7f98c7533700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:31:39.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.279+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e9c13fb60 con 0x7f0e80066f30 2023-10-06T04:31:39.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.287+0000 7f0e917fa700 1 -- 172.21.15.123:0/1814740268 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286715 (secure 0 0 0) 0x7f0e9800baf0 con 0x7f0e80066f30 2023-10-06T04:31:39.292 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:39.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e80066f30 msgr2=0x7f0e800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e80066f30 0x7f0e800693e0 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f0e9c0cc610 tx=0x7f0e98016000 comp rx=0 tx=0).stop 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c068bb0 msgr2=0x7f0e9c1542b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c068bb0 0x7f0e9c1542b0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0e8c00b600 tx=0x7f0e8c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 shutdown_connections 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e80066f30 0x7f0e800693e0 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e9c068bb0 0x7f0e9c1542b0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e9c1597e0 0x7f0e9c156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 --2- 172.21.15.123:0/1814740268 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e9c159410 0x7f0e9c13e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 >> 172.21.15.123:0/1814740268 conn(0x7f0e9c0c9c90 msgr2=0x7f0e9c0b1e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 shutdown_connections 2023-10-06T04:31:39.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.290+0000 7f0ea3842700 1 -- 172.21.15.123:0/1814740268 wait complete. 2023-10-06T04:31:39.296 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:31:39.303 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-06T04:31:38.882081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381280,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593440,"statfs":{"total":773094113280,"available":770655682560,"internally_reserved":0,"allocated":3047424,"data_stored":1843020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.859937"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057310+0000","last_change":"2023-10-06T04:31:35.057310+0000","last_active":"2023-10-06T04:31:35.057310+0000","last_peered":"2023-10-06T04:31:35.057310+0000","last_clean":"2023-10-06T04:31:35.057310+0000","last_became_active":"2023-10-06T04:31:35.057042+0000","last_became_peered":"2023-10-06T04:31:35.057042+0000","last_unstale":"2023-10-06T04:31:35.057310+0000","last_undegraded":"2023-10-06T04:31:35.057310+0000","last_fullsized":"2023-10-06T04:31:35.057310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063658+0000","last_change":"2023-10-06T04:31:35.063658+0000","last_active":"2023-10-06T04:31:35.063658+0000","last_peered":"2023-10-06T04:31:35.063658+0000","last_clean":"2023-10-06T04:31:35.063658+0000","last_became_active":"2023-10-06T04:31:35.063403+0000","last_became_peered":"2023-10-06T04:31:35.063403+0000","last_unstale":"2023-10-06T04:31:35.063658+0000","last_undegraded":"2023-10-06T04:31:35.063658+0000","last_fullsized":"2023-10-06T04:31:35.063658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.056211+0000","last_change":"2023-10-06T04:31:35.056211+0000","last_active":"2023-10-06T04:31:35.056211+0000","last_peered":"2023-10-06T04:31:35.056211+0000","last_clean":"2023-10-06T04:31:35.056211+0000","last_became_active":"2023-10-06T04:31:35.055883+0000","last_became_peered":"2023-10-06T04:31:35.055883+0000","last_unstale":"2023-10-06T04:31:35.056211+0000","last_undegraded":"2023-10-06T04:31:35.056211+0000","last_fullsized":"2023-10-06T04:31:35.056211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:54.893111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055601+0000","last_change":"2023-10-06T04:31:35.055601+0000","last_active":"2023-10-06T04:31:35.055601+0000","last_peered":"2023-10-06T04:31:35.055601+0000","last_clean":"2023-10-06T04:31:35.055601+0000","last_became_active":"2023-10-06T04:31:35.055325+0000","last_became_peered":"2023-10-06T04:31:35.055325+0000","last_unstale":"2023-10-06T04:31:35.055601+0000","last_undegraded":"2023-10-06T04:31:35.055601+0000","last_fullsized":"2023-10-06T04:31:35.055601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061888+0000","last_change":"2023-10-06T04:31:35.061888+0000","last_active":"2023-10-06T04:31:35.061888+0000","last_peered":"2023-10-06T04:31:35.061888+0000","last_clean":"2023-10-06T04:31:35.061888+0000","last_became_active":"2023-10-06T04:31:35.061635+0000","last_became_peered":"2023-10-06T04:31:35.061635+0000","last_unstale":"2023-10-06T04:31:35.061888+0000","last_undegraded":"2023-10-06T04:31:35.061888+0000","last_fullsized":"2023-10-06T04:31:35.061888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055786+0000","last_change":"2023-10-06T04:31:35.055786+0000","last_active":"2023-10-06T04:31:35.055786+0000","last_peered":"2023-10-06T04:31:35.055786+0000","last_clean":"2023-10-06T04:31:35.055786+0000","last_became_active":"2023-10-06T04:31:35.055501+0000","last_became_peered":"2023-10-06T04:31:35.055501+0000","last_unstale":"2023-10-06T04:31:35.055786+0000","last_undegraded":"2023-10-06T04:31:35.055786+0000","last_fullsized":"2023-10-06T04:31:35.055786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:47.521895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065611+0000","last_change":"2023-10-06T04:31:35.065611+0000","last_active":"2023-10-06T04:31:35.065611+0000","last_peered":"2023-10-06T04:31:35.065611+0000","last_clean":"2023-10-06T04:31:35.065611+0000","last_became_active":"2023-10-06T04:31:35.064864+0000","last_became_peered":"2023-10-06T04:31:35.064864+0000","last_unstale":"2023-10-06T04:31:35.065611+0000","last_undegraded":"2023-10-06T04:31:35.065611+0000","last_fullsized":"2023-10-06T04:31:35.065611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060662+0000","last_change":"2023-10-06T04:31:35.060662+0000","last_active":"2023-10-06T04:31:35.060662+0000","last_peered":"2023-10-06T04:31:35.060662+0000","last_clean":"2023-10-06T04:31:35.060662+0000","last_became_active":"2023-10-06T04:31:35.060371+0000","last_became_peered":"2023-10-06T04:31:35.060371+0000","last_unstale":"2023-10-06T04:31:35.060662+0000","last_undegraded":"2023-10-06T04:31:35.060662+0000","last_fullsized":"2023-10-06T04:31:35.060662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061068+0000","last_change":"2023-10-06T04:31:35.061068+0000","last_active":"2023-10-06T04:31:35.061068+0000","last_peered":"2023-10-06T04:31:35.061068+0000","last_clean":"2023-10-06T04:31:35.061068+0000","last_became_active":"2023-10-06T04:31:35.060663+0000","last_became_peered":"2023-10-06T04:31:35.060663+0000","last_unstale":"2023-10-06T04:31:35.061068+0000","last_undegraded":"2023-10-06T04:31:35.061068+0000","last_fullsized":"2023-10-06T04:31:35.061068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062498+0000","last_change":"2023-10-06T04:31:35.062498+0000","last_active":"2023-10-06T04:31:35.062498+0000","last_peered":"2023-10-06T04:31:35.062498+0000","last_clean":"2023-10-06T04:31:35.062498+0000","last_became_active":"2023-10-06T04:31:35.062236+0000","last_became_peered":"2023-10-06T04:31:35.062236+0000","last_unstale":"2023-10-06T04:31:35.062498+0000","last_undegraded":"2023-10-06T04:31:35.062498+0000","last_fullsized":"2023-10-06T04:31:35.062498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057945+0000","last_change":"2023-10-06T04:31:35.057945+0000","last_active":"2023-10-06T04:31:35.057945+0000","last_peered":"2023-10-06T04:31:35.057945+0000","last_clean":"2023-10-06T04:31:35.057945+0000","last_became_active":"2023-10-06T04:31:35.057568+0000","last_became_peered":"2023-10-06T04:31:35.057568+0000","last_unstale":"2023-10-06T04:31:35.057945+0000","last_undegraded":"2023-10-06T04:31:35.057945+0000","last_fullsized":"2023-10-06T04:31:35.057945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:55.588795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051600+0000","last_change":"2023-10-06T04:31:35.051600+0000","last_active":"2023-10-06T04:31:35.051600+0000","last_peered":"2023-10-06T04:31:35.051600+0000","last_clean":"2023-10-06T04:31:35.051600+0000","last_became_active":"2023-10-06T04:31:35.051344+0000","last_became_peered":"2023-10-06T04:31:35.051344+0000","last_unstale":"2023-10-06T04:31:35.051600+0000","last_undegraded":"2023-10-06T04:31:35.051600+0000","last_fullsized":"2023-10-06T04:31:35.051600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055073+0000","last_change":"2023-10-06T04:31:35.055073+0000","last_active":"2023-10-06T04:31:35.055073+0000","last_peered":"2023-10-06T04:31:35.055073+0000","last_clean":"2023-10-06T04:31:35.055073+0000","last_became_active":"2023-10-06T04:31:35.054723+0000","last_became_peered":"2023-10-06T04:31:35.054723+0000","last_unstale":"2023-10-06T04:31:35.055073+0000","last_undegraded":"2023-10-06T04:31:35.055073+0000","last_fullsized":"2023-10-06T04:31:35.055073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:53.494026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065191+0000","last_change":"2023-10-06T04:31:35.065191+0000","last_active":"2023-10-06T04:31:35.065191+0000","last_peered":"2023-10-06T04:31:35.065191+0000","last_clean":"2023-10-06T04:31:35.065191+0000","last_became_active":"2023-10-06T04:31:35.064865+0000","last_became_peered":"2023-10-06T04:31:35.064865+0000","last_unstale":"2023-10-06T04:31:35.065191+0000","last_undegraded":"2023-10-06T04:31:35.065191+0000","last_fullsized":"2023-10-06T04:31:35.065191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065835+0000","last_change":"2023-10-06T04:31:35.065835+0000","last_active":"2023-10-06T04:31:35.065835+0000","last_peered":"2023-10-06T04:31:35.065835+0000","last_clean":"2023-10-06T04:31:35.065835+0000","last_became_active":"2023-10-06T04:31:35.065225+0000","last_became_peered":"2023-10-06T04:31:35.065225+0000","last_unstale":"2023-10-06T04:31:35.065835+0000","last_undegraded":"2023-10-06T04:31:35.065835+0000","last_fullsized":"2023-10-06T04:31:35.065835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050260+0000","last_change":"2023-10-06T04:31:35.050260+0000","last_active":"2023-10-06T04:31:35.050260+0000","last_peered":"2023-10-06T04:31:35.050260+0000","last_clean":"2023-10-06T04:31:35.050260+0000","last_became_active":"2023-10-06T04:31:35.049943+0000","last_became_peered":"2023-10-06T04:31:35.049943+0000","last_unstale":"2023-10-06T04:31:35.050260+0000","last_undegraded":"2023-10-06T04:31:35.050260+0000","last_fullsized":"2023-10-06T04:31:35.050260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060912+0000","last_change":"2023-10-06T04:31:35.060912+0000","last_active":"2023-10-06T04:31:35.060912+0000","last_peered":"2023-10-06T04:31:35.060912+0000","last_clean":"2023-10-06T04:31:35.060912+0000","last_became_active":"2023-10-06T04:31:35.060330+0000","last_became_peered":"2023-10-06T04:31:35.060330+0000","last_unstale":"2023-10-06T04:31:35.060912+0000","last_undegraded":"2023-10-06T04:31:35.060912+0000","last_fullsized":"2023-10-06T04:31:35.060912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057646+0000","last_change":"2023-10-06T04:31:35.057646+0000","last_active":"2023-10-06T04:31:35.057646+0000","last_peered":"2023-10-06T04:31:35.057646+0000","last_clean":"2023-10-06T04:31:35.057646+0000","last_became_active":"2023-10-06T04:31:35.057375+0000","last_became_peered":"2023-10-06T04:31:35.057375+0000","last_unstale":"2023-10-06T04:31:35.057646+0000","last_undegraded":"2023-10-06T04:31:35.057646+0000","last_fullsized":"2023-10-06T04:31:35.057646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:44.450170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046638+0000","last_change":"2023-10-06T04:31:35.046638+0000","last_active":"2023-10-06T04:31:35.046638+0000","last_peered":"2023-10-06T04:31:35.046638+0000","last_clean":"2023-10-06T04:31:35.046638+0000","last_became_active":"2023-10-06T04:31:35.046376+0000","last_became_peered":"2023-10-06T04:31:35.046376+0000","last_unstale":"2023-10-06T04:31:35.046638+0000","last_undegraded":"2023-10-06T04:31:35.046638+0000","last_fullsized":"2023-10-06T04:31:35.046638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:33.326007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052597+0000","last_change":"2023-10-06T04:31:35.052597+0000","last_active":"2023-10-06T04:31:35.052597+0000","last_peered":"2023-10-06T04:31:35.052597+0000","last_clean":"2023-10-06T04:31:35.052597+0000","last_became_active":"2023-10-06T04:31:35.052318+0000","last_became_peered":"2023-10-06T04:31:35.052318+0000","last_unstale":"2023-10-06T04:31:35.052597+0000","last_undegraded":"2023-10-06T04:31:35.052597+0000","last_fullsized":"2023-10-06T04:31:35.052597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.067001+0000","last_change":"2023-10-06T04:31:35.067001+0000","last_active":"2023-10-06T04:31:35.067001+0000","last_peered":"2023-10-06T04:31:35.067001+0000","last_clean":"2023-10-06T04:31:35.067001+0000","last_became_active":"2023-10-06T04:31:35.066763+0000","last_became_peered":"2023-10-06T04:31:35.066763+0000","last_unstale":"2023-10-06T04:31:35.067001+0000","last_undegraded":"2023-10-06T04:31:35.067001+0000","last_fullsized":"2023-10-06T04:31:35.067001+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059383+0000","last_change":"2023-10-06T04:31:35.059383+0000","last_active":"2023-10-06T04:31:35.059383+0000","last_peered":"2023-10-06T04:31:35.059383+0000","last_clean":"2023-10-06T04:31:35.059383+0000","last_became_active":"2023-10-06T04:31:35.059106+0000","last_became_peered":"2023-10-06T04:31:35.059106+0000","last_unstale":"2023-10-06T04:31:35.059383+0000","last_undegraded":"2023-10-06T04:31:35.059383+0000","last_fullsized":"2023-10-06T04:31:35.059383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054733+0000","last_change":"2023-10-06T04:31:35.054733+0000","last_active":"2023-10-06T04:31:35.054733+0000","last_peered":"2023-10-06T04:31:35.054733+0000","last_clean":"2023-10-06T04:31:35.054733+0000","last_became_active":"2023-10-06T04:31:35.054454+0000","last_became_peered":"2023-10-06T04:31:35.054454+0000","last_unstale":"2023-10-06T04:31:35.054733+0000","last_undegraded":"2023-10-06T04:31:35.054733+0000","last_fullsized":"2023-10-06T04:31:35.054733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059648+0000","last_change":"2023-10-06T04:31:35.059648+0000","last_active":"2023-10-06T04:31:35.059648+0000","last_peered":"2023-10-06T04:31:35.059648+0000","last_clean":"2023-10-06T04:31:35.059648+0000","last_became_active":"2023-10-06T04:31:35.059380+0000","last_became_peered":"2023-10-06T04:31:35.059380+0000","last_unstale":"2023-10-06T04:31:35.059648+0000","last_undegraded":"2023-10-06T04:31:35.059648+0000","last_fullsized":"2023-10-06T04:31:35.059648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063285+0000","last_change":"2023-10-06T04:31:35.063285+0000","last_active":"2023-10-06T04:31:35.063285+0000","last_peered":"2023-10-06T04:31:35.063285+0000","last_clean":"2023-10-06T04:31:35.063285+0000","last_became_active":"2023-10-06T04:31:35.062975+0000","last_became_peered":"2023-10-06T04:31:35.062975+0000","last_unstale":"2023-10-06T04:31:35.063285+0000","last_undegraded":"2023-10-06T04:31:35.063285+0000","last_fullsized":"2023-10-06T04:31:35.063285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058231+0000","last_change":"2023-10-06T04:31:35.058231+0000","last_active":"2023-10-06T04:31:35.058231+0000","last_peered":"2023-10-06T04:31:35.058231+0000","last_clean":"2023-10-06T04:31:35.058231+0000","last_became_active":"2023-10-06T04:31:35.057944+0000","last_became_peered":"2023-10-06T04:31:35.057944+0000","last_unstale":"2023-10-06T04:31:35.058231+0000","last_undegraded":"2023-10-06T04:31:35.058231+0000","last_fullsized":"2023-10-06T04:31:35.058231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063120+0000","last_change":"2023-10-06T04:31:35.063120+0000","last_active":"2023-10-06T04:31:35.063120+0000","last_peered":"2023-10-06T04:31:35.063120+0000","last_clean":"2023-10-06T04:31:35.063120+0000","last_became_active":"2023-10-06T04:31:35.062868+0000","last_became_peered":"2023-10-06T04:31:35.062868+0000","last_unstale":"2023-10-06T04:31:35.063120+0000","last_undegraded":"2023-10-06T04:31:35.063120+0000","last_fullsized":"2023-10-06T04:31:35.063120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046895+0000","last_change":"2023-10-06T04:31:34.042620+0000","last_active":"2023-10-06T04:31:35.046895+0000","last_peered":"2023-10-06T04:31:35.046895+0000","last_clean":"2023-10-06T04:31:35.046895+0000","last_became_active":"2023-10-06T04:31:34.042023+0000","last_became_peered":"2023-10-06T04:31:34.042023+0000","last_unstale":"2023-10-06T04:31:35.046895+0000","last_undegraded":"2023-10-06T04:31:35.046895+0000","last_fullsized":"2023-10-06T04:31:35.046895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060955+0000","last_change":"2023-10-06T04:31:35.060955+0000","last_active":"2023-10-06T04:31:35.060955+0000","last_peered":"2023-10-06T04:31:35.060955+0000","last_clean":"2023-10-06T04:31:35.060955+0000","last_became_active":"2023-10-06T04:31:35.060124+0000","last_became_peered":"2023-10-06T04:31:35.060124+0000","last_unstale":"2023-10-06T04:31:35.060955+0000","last_undegraded":"2023-10-06T04:31:35.060955+0000","last_fullsized":"2023-10-06T04:31:35.060955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.040490+0000","last_change":"2023-10-06T04:31:34.036811+0000","last_active":"2023-10-06T04:31:34.040490+0000","last_peered":"2023-10-06T04:31:34.040490+0000","last_clean":"2023-10-06T04:31:34.040490+0000","last_became_active":"2023-10-06T04:31:34.036513+0000","last_became_peered":"2023-10-06T04:31:34.036513+0000","last_unstale":"2023-10-06T04:31:34.040490+0000","last_undegraded":"2023-10-06T04:31:34.040490+0000","last_fullsized":"2023-10-06T04:31:34.040490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055379+0000","last_change":"2023-10-06T04:31:35.055379+0000","last_active":"2023-10-06T04:31:35.055379+0000","last_peered":"2023-10-06T04:31:35.055379+0000","last_clean":"2023-10-06T04:31:35.055379+0000","last_became_active":"2023-10-06T04:31:35.055088+0000","last_became_peered":"2023-10-06T04:31:35.055088+0000","last_unstale":"2023-10-06T04:31:35.055379+0000","last_undegraded":"2023-10-06T04:31:35.055379+0000","last_fullsized":"2023-10-06T04:31:35.055379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.057492+0000","last_change":"2023-10-06T04:31:34.054496+0000","last_active":"2023-10-06T04:31:34.057492+0000","last_peered":"2023-10-06T04:31:34.057492+0000","last_clean":"2023-10-06T04:31:34.057492+0000","last_became_active":"2023-10-06T04:31:34.054225+0000","last_became_peered":"2023-10-06T04:31:34.054225+0000","last_unstale":"2023-10-06T04:31:34.057492+0000","last_undegraded":"2023-10-06T04:31:34.057492+0000","last_fullsized":"2023-10-06T04:31:34.057492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058805+0000","last_change":"2023-10-06T04:31:35.058805+0000","last_active":"2023-10-06T04:31:35.058805+0000","last_peered":"2023-10-06T04:31:35.058805+0000","last_clean":"2023-10-06T04:31:35.058805+0000","last_became_active":"2023-10-06T04:31:35.058524+0000","last_became_peered":"2023-10-06T04:31:35.058524+0000","last_unstale":"2023-10-06T04:31:35.058805+0000","last_undegraded":"2023-10-06T04:31:35.058805+0000","last_fullsized":"2023-10-06T04:31:35.058805+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051512+0000","last_change":"2023-10-06T04:31:34.042810+0000","last_active":"2023-10-06T04:31:34.051512+0000","last_peered":"2023-10-06T04:31:34.051512+0000","last_clean":"2023-10-06T04:31:34.051512+0000","last_became_active":"2023-10-06T04:31:34.040519+0000","last_became_peered":"2023-10-06T04:31:34.040519+0000","last_unstale":"2023-10-06T04:31:34.051512+0000","last_undegraded":"2023-10-06T04:31:34.051512+0000","last_fullsized":"2023-10-06T04:31:34.051512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061039+0000","last_change":"2023-10-06T04:31:35.061039+0000","last_active":"2023-10-06T04:31:35.061039+0000","last_peered":"2023-10-06T04:31:35.061039+0000","last_clean":"2023-10-06T04:31:35.061039+0000","last_became_active":"2023-10-06T04:31:35.060281+0000","last_became_peered":"2023-10-06T04:31:35.060281+0000","last_unstale":"2023-10-06T04:31:35.061039+0000","last_undegraded":"2023-10-06T04:31:35.061039+0000","last_fullsized":"2023-10-06T04:31:35.061039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.047409+0000","last_change":"2023-10-06T04:31:34.039478+0000","last_active":"2023-10-06T04:31:34.047409+0000","last_peered":"2023-10-06T04:31:34.047409+0000","last_clean":"2023-10-06T04:31:34.047409+0000","last_became_active":"2023-10-06T04:31:34.039209+0000","last_became_peered":"2023-10-06T04:31:34.039209+0000","last_unstale":"2023-10-06T04:31:34.047409+0000","last_undegraded":"2023-10-06T04:31:34.047409+0000","last_fullsized":"2023-10-06T04:31:34.047409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063717+0000","last_change":"2023-10-06T04:31:35.063717+0000","last_active":"2023-10-06T04:31:35.063717+0000","last_peered":"2023-10-06T04:31:35.063717+0000","last_clean":"2023-10-06T04:31:35.063717+0000","last_became_active":"2023-10-06T04:31:35.060241+0000","last_became_peered":"2023-10-06T04:31:35.060241+0000","last_unstale":"2023-10-06T04:31:35.063717+0000","last_undegraded":"2023-10-06T04:31:35.063717+0000","last_fullsized":"2023-10-06T04:31:35.063717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.252505+0000","last_change":"2023-10-06T04:31:34.045777+0000","last_active":"2023-10-06T04:31:35.252505+0000","last_peered":"2023-10-06T04:31:35.252505+0000","last_clean":"2023-10-06T04:31:35.252505+0000","last_became_active":"2023-10-06T04:31:34.045498+0000","last_became_peered":"2023-10-06T04:31:34.045498+0000","last_unstale":"2023-10-06T04:31:35.252505+0000","last_undegraded":"2023-10-06T04:31:35.252505+0000","last_fullsized":"2023-10-06T04:31:35.252505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.139941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063439+0000","last_change":"2023-10-06T04:31:35.063439+0000","last_active":"2023-10-06T04:31:35.063439+0000","last_peered":"2023-10-06T04:31:35.063439+0000","last_clean":"2023-10-06T04:31:35.063439+0000","last_became_active":"2023-10-06T04:31:35.063117+0000","last_became_peered":"2023-10-06T04:31:35.063117+0000","last_unstale":"2023-10-06T04:31:35.063439+0000","last_undegraded":"2023-10-06T04:31:35.063439+0000","last_fullsized":"2023-10-06T04:31:35.063439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:16.144634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257849+0000","last_change":"2023-10-06T04:31:34.039091+0000","last_active":"2023-10-06T04:31:35.257849+0000","last_peered":"2023-10-06T04:31:35.257849+0000","last_clean":"2023-10-06T04:31:35.257849+0000","last_became_active":"2023-10-06T04:31:34.038778+0000","last_became_peered":"2023-10-06T04:31:34.038778+0000","last_unstale":"2023-10-06T04:31:35.257849+0000","last_undegraded":"2023-10-06T04:31:35.257849+0000","last_fullsized":"2023-10-06T04:31:35.257849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052851+0000","last_change":"2023-10-06T04:31:35.052851+0000","last_active":"2023-10-06T04:31:35.052851+0000","last_peered":"2023-10-06T04:31:35.052851+0000","last_clean":"2023-10-06T04:31:35.052851+0000","last_became_active":"2023-10-06T04:31:35.052560+0000","last_became_peered":"2023-10-06T04:31:35.052560+0000","last_unstale":"2023-10-06T04:31:35.052851+0000","last_undegraded":"2023-10-06T04:31:35.052851+0000","last_fullsized":"2023-10-06T04:31:35.052851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:39.105270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.254917+0000","last_change":"2023-10-06T04:31:34.046053+0000","last_active":"2023-10-06T04:31:35.254917+0000","last_peered":"2023-10-06T04:31:35.254917+0000","last_clean":"2023-10-06T04:31:35.254917+0000","last_became_active":"2023-10-06T04:31:34.045776+0000","last_became_peered":"2023-10-06T04:31:34.045776+0000","last_unstale":"2023-10-06T04:31:35.254917+0000","last_undegraded":"2023-10-06T04:31:35.254917+0000","last_fullsized":"2023-10-06T04:31:35.254917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:40.537300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062504+0000","last_change":"2023-10-06T04:31:35.062504+0000","last_active":"2023-10-06T04:31:35.062504+0000","last_peered":"2023-10-06T04:31:35.062504+0000","last_clean":"2023-10-06T04:31:35.062504+0000","last_became_active":"2023-10-06T04:31:35.060884+0000","last_became_peered":"2023-10-06T04:31:35.060884+0000","last_unstale":"2023-10-06T04:31:35.062504+0000","last_undegraded":"2023-10-06T04:31:35.062504+0000","last_fullsized":"2023-10-06T04:31:35.062504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051578+0000","last_change":"2023-10-06T04:31:34.042234+0000","last_active":"2023-10-06T04:31:34.051578+0000","last_peered":"2023-10-06T04:31:34.051578+0000","last_clean":"2023-10-06T04:31:34.051578+0000","last_became_active":"2023-10-06T04:31:34.041959+0000","last_became_peered":"2023-10-06T04:31:34.041959+0000","last_unstale":"2023-10-06T04:31:34.051578+0000","last_undegraded":"2023-10-06T04:31:34.051578+0000","last_fullsized":"2023-10-06T04:31:34.051578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061678+0000","last_change":"2023-10-06T04:31:35.061678+0000","last_active":"2023-10-06T04:31:35.061678+0000","last_peered":"2023-10-06T04:31:35.061678+0000","last_clean":"2023-10-06T04:31:35.061678+0000","last_became_active":"2023-10-06T04:31:35.061383+0000","last_became_peered":"2023-10-06T04:31:35.061383+0000","last_unstale":"2023-10-06T04:31:35.061678+0000","last_undegraded":"2023-10-06T04:31:35.061678+0000","last_fullsized":"2023-10-06T04:31:35.061678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.048210+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:34.048210+0000","last_peered":"2023-10-06T04:31:34.048210+0000","last_clean":"2023-10-06T04:31:34.048210+0000","last_became_active":"2023-10-06T04:31:34.046107+0000","last_became_peered":"2023-10-06T04:31:34.046107+0000","last_unstale":"2023-10-06T04:31:34.048210+0000","last_undegraded":"2023-10-06T04:31:34.048210+0000","last_fullsized":"2023-10-06T04:31:34.048210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055168+0000","last_change":"2023-10-06T04:31:35.055168+0000","last_active":"2023-10-06T04:31:35.055168+0000","last_peered":"2023-10-06T04:31:35.055168+0000","last_clean":"2023-10-06T04:31:35.055168+0000","last_became_active":"2023-10-06T04:31:35.054903+0000","last_became_peered":"2023-10-06T04:31:35.054903+0000","last_unstale":"2023-10-06T04:31:35.055168+0000","last_undegraded":"2023-10-06T04:31:35.055168+0000","last_fullsized":"2023-10-06T04:31:35.055168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257431+0000","last_change":"2023-10-06T04:31:34.039233+0000","last_active":"2023-10-06T04:31:35.257431+0000","last_peered":"2023-10-06T04:31:35.257431+0000","last_clean":"2023-10-06T04:31:35.257431+0000","last_became_active":"2023-10-06T04:31:34.038916+0000","last_became_peered":"2023-10-06T04:31:34.038916+0000","last_unstale":"2023-10-06T04:31:35.257431+0000","last_undegraded":"2023-10-06T04:31:35.257431+0000","last_fullsized":"2023-10-06T04:31:35.257431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:31.215430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062145+0000","last_change":"2023-10-06T04:31:35.062145+0000","last_active":"2023-10-06T04:31:35.062145+0000","last_peered":"2023-10-06T04:31:35.062145+0000","last_clean":"2023-10-06T04:31:35.062145+0000","last_became_active":"2023-10-06T04:31:35.061180+0000","last_became_peered":"2023-10-06T04:31:35.061180+0000","last_unstale":"2023-10-06T04:31:35.062145+0000","last_undegraded":"2023-10-06T04:31:35.062145+0000","last_fullsized":"2023-10-06T04:31:35.062145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050141+0000","last_change":"2023-10-06T04:31:34.043099+0000","last_active":"2023-10-06T04:31:35.050141+0000","last_peered":"2023-10-06T04:31:35.050141+0000","last_clean":"2023-10-06T04:31:35.050141+0000","last_became_active":"2023-10-06T04:31:34.042869+0000","last_became_peered":"2023-10-06T04:31:34.042869+0000","last_unstale":"2023-10-06T04:31:35.050141+0000","last_undegraded":"2023-10-06T04:31:35.050141+0000","last_fullsized":"2023-10-06T04:31:35.050141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061606+0000","last_change":"2023-10-06T04:31:35.061606+0000","last_active":"2023-10-06T04:31:35.061606+0000","last_peered":"2023-10-06T04:31:35.061606+0000","last_clean":"2023-10-06T04:31:35.061606+0000","last_became_active":"2023-10-06T04:31:35.061333+0000","last_became_peered":"2023-10-06T04:31:35.061333+0000","last_unstale":"2023-10-06T04:31:35.061606+0000","last_undegraded":"2023-10-06T04:31:35.061606+0000","last_fullsized":"2023-10-06T04:31:35.061606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255026+0000","last_change":"2023-10-06T04:31:34.045981+0000","last_active":"2023-10-06T04:31:35.255026+0000","last_peered":"2023-10-06T04:31:35.255026+0000","last_clean":"2023-10-06T04:31:35.255026+0000","last_became_active":"2023-10-06T04:31:34.045683+0000","last_became_peered":"2023-10-06T04:31:34.045683+0000","last_unstale":"2023-10-06T04:31:35.255026+0000","last_undegraded":"2023-10-06T04:31:35.255026+0000","last_fullsized":"2023-10-06T04:31:35.255026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:22.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059429+0000","last_change":"2023-10-06T04:31:35.059429+0000","last_active":"2023-10-06T04:31:35.059429+0000","last_peered":"2023-10-06T04:31:35.059429+0000","last_clean":"2023-10-06T04:31:35.059429+0000","last_became_active":"2023-10-06T04:31:35.058829+0000","last_became_peered":"2023-10-06T04:31:35.058829+0000","last_unstale":"2023-10-06T04:31:35.059429+0000","last_undegraded":"2023-10-06T04:31:35.059429+0000","last_fullsized":"2023-10-06T04:31:35.059429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049513+0000","last_change":"2023-10-06T04:31:34.048038+0000","last_active":"2023-10-06T04:31:35.049513+0000","last_peered":"2023-10-06T04:31:35.049513+0000","last_clean":"2023-10-06T04:31:35.049513+0000","last_became_active":"2023-10-06T04:31:34.047780+0000","last_became_peered":"2023-10-06T04:31:34.047780+0000","last_unstale":"2023-10-06T04:31:35.049513+0000","last_undegraded":"2023-10-06T04:31:35.049513+0000","last_fullsized":"2023-10-06T04:31:35.049513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:21.519366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057281+0000","last_change":"2023-10-06T04:31:35.057281+0000","last_active":"2023-10-06T04:31:35.057281+0000","last_peered":"2023-10-06T04:31:35.057281+0000","last_clean":"2023-10-06T04:31:35.057281+0000","last_became_active":"2023-10-06T04:31:35.056855+0000","last_became_peered":"2023-10-06T04:31:35.056855+0000","last_unstale":"2023-10-06T04:31:35.057281+0000","last_undegraded":"2023-10-06T04:31:35.057281+0000","last_fullsized":"2023-10-06T04:31:35.057281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:23.765556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.048027+0000","last_change":"2023-10-06T04:31:34.046008+0000","last_active":"2023-10-06T04:31:34.048027+0000","last_peered":"2023-10-06T04:31:34.048027+0000","last_clean":"2023-10-06T04:31:34.048027+0000","last_became_active":"2023-10-06T04:31:34.045732+0000","last_became_peered":"2023-10-06T04:31:34.045732+0000","last_unstale":"2023-10-06T04:31:34.048027+0000","last_undegraded":"2023-10-06T04:31:34.048027+0000","last_fullsized":"2023-10-06T04:31:34.048027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059341+0000","last_change":"2023-10-06T04:31:35.059341+0000","last_active":"2023-10-06T04:31:35.059341+0000","last_peered":"2023-10-06T04:31:35.059341+0000","last_clean":"2023-10-06T04:31:35.059341+0000","last_became_active":"2023-10-06T04:31:35.059087+0000","last_became_peered":"2023-10-06T04:31:35.059087+0000","last_unstale":"2023-10-06T04:31:35.059341+0000","last_undegraded":"2023-10-06T04:31:35.059341+0000","last_fullsized":"2023-10-06T04:31:35.059341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.249360+0000","last_change":"2023-10-06T04:31:34.045304+0000","last_active":"2023-10-06T04:31:35.249360+0000","last_peered":"2023-10-06T04:31:35.249360+0000","last_clean":"2023-10-06T04:31:35.249360+0000","last_became_active":"2023-10-06T04:31:34.044891+0000","last_became_peered":"2023-10-06T04:31:34.044891+0000","last_unstale":"2023-10-06T04:31:35.249360+0000","last_undegraded":"2023-10-06T04:31:35.249360+0000","last_fullsized":"2023-10-06T04:31:35.249360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055035+0000","last_change":"2023-10-06T04:31:35.055035+0000","last_active":"2023-10-06T04:31:35.055035+0000","last_peered":"2023-10-06T04:31:35.055035+0000","last_clean":"2023-10-06T04:31:35.055035+0000","last_became_active":"2023-10-06T04:31:35.054727+0000","last_became_peered":"2023-10-06T04:31:35.054727+0000","last_unstale":"2023-10-06T04:31:35.055035+0000","last_undegraded":"2023-10-06T04:31:35.055035+0000","last_fullsized":"2023-10-06T04:31:35.055035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:31.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.046849+0000","last_change":"2023-10-06T04:31:34.041723+0000","last_active":"2023-10-06T04:31:34.046849+0000","last_peered":"2023-10-06T04:31:34.046849+0000","last_clean":"2023-10-06T04:31:34.046849+0000","last_became_active":"2023-10-06T04:31:34.041386+0000","last_became_peered":"2023-10-06T04:31:34.041386+0000","last_unstale":"2023-10-06T04:31:34.046849+0000","last_undegraded":"2023-10-06T04:31:34.046849+0000","last_fullsized":"2023-10-06T04:31:34.046849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:58.993061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057831+0000","last_change":"2023-10-06T04:31:35.057831+0000","last_active":"2023-10-06T04:31:35.057831+0000","last_peered":"2023-10-06T04:31:35.057831+0000","last_clean":"2023-10-06T04:31:35.057831+0000","last_became_active":"2023-10-06T04:31:35.057257+0000","last_became_peered":"2023-10-06T04:31:35.057257+0000","last_unstale":"2023-10-06T04:31:35.057831+0000","last_undegraded":"2023-10-06T04:31:35.057831+0000","last_fullsized":"2023-10-06T04:31:35.057831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.040032+0000","last_change":"2023-10-06T04:31:34.035016+0000","last_active":"2023-10-06T04:31:34.040032+0000","last_peered":"2023-10-06T04:31:34.040032+0000","last_clean":"2023-10-06T04:31:34.040032+0000","last_became_active":"2023-10-06T04:31:34.034721+0000","last_became_peered":"2023-10-06T04:31:34.034721+0000","last_unstale":"2023-10-06T04:31:34.040032+0000","last_undegraded":"2023-10-06T04:31:34.040032+0000","last_fullsized":"2023-10-06T04:31:34.040032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049984+0000","last_change":"2023-10-06T04:31:35.049984+0000","last_active":"2023-10-06T04:31:35.049984+0000","last_peered":"2023-10-06T04:31:35.049984+0000","last_clean":"2023-10-06T04:31:35.049984+0000","last_became_active":"2023-10-06T04:31:35.049687+0000","last_became_peered":"2023-10-06T04:31:35.049687+0000","last_unstale":"2023-10-06T04:31:35.049984+0000","last_undegraded":"2023-10-06T04:31:35.049984+0000","last_fullsized":"2023-10-06T04:31:35.049984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051973+0000","last_change":"2023-10-06T04:31:34.046626+0000","last_active":"2023-10-06T04:31:34.051973+0000","last_peered":"2023-10-06T04:31:34.051973+0000","last_clean":"2023-10-06T04:31:34.051973+0000","last_became_active":"2023-10-06T04:31:34.044656+0000","last_became_peered":"2023-10-06T04:31:34.044656+0000","last_unstale":"2023-10-06T04:31:34.051973+0000","last_undegraded":"2023-10-06T04:31:34.051973+0000","last_fullsized":"2023-10-06T04:31:34.051973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059496+0000","last_change":"2023-10-06T04:31:35.059496+0000","last_active":"2023-10-06T04:31:35.059496+0000","last_peered":"2023-10-06T04:31:35.059496+0000","last_clean":"2023-10-06T04:31:35.059496+0000","last_became_active":"2023-10-06T04:31:35.059187+0000","last_became_peered":"2023-10-06T04:31:35.059187+0000","last_unstale":"2023-10-06T04:31:35.059496+0000","last_undegraded":"2023-10-06T04:31:35.059496+0000","last_fullsized":"2023-10-06T04:31:35.059496+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.037449+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:31:34.037449+0000","last_peered":"2023-10-06T04:31:34.037449+0000","last_clean":"2023-10-06T04:31:34.037449+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:31:34.037449+0000","last_undegraded":"2023-10-06T04:31:34.037449+0000","last_fullsized":"2023-10-06T04:31:34.037449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.040196+0000","last_change":"2023-10-06T04:31:34.036361+0000","last_active":"2023-10-06T04:31:34.040196+0000","last_peered":"2023-10-06T04:31:34.040196+0000","last_clean":"2023-10-06T04:31:34.040196+0000","last_became_active":"2023-10-06T04:31:34.036048+0000","last_became_peered":"2023-10-06T04:31:34.036048+0000","last_unstale":"2023-10-06T04:31:34.040196+0000","last_undegraded":"2023-10-06T04:31:34.040196+0000","last_fullsized":"2023-10-06T04:31:34.040196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061294+0000","last_change":"2023-10-06T04:31:35.061294+0000","last_active":"2023-10-06T04:31:35.061294+0000","last_peered":"2023-10-06T04:31:35.061294+0000","last_clean":"2023-10-06T04:31:35.061294+0000","last_became_active":"2023-10-06T04:31:35.061034+0000","last_became_peered":"2023-10-06T04:31:35.061034+0000","last_unstale":"2023-10-06T04:31:35.061294+0000","last_undegraded":"2023-10-06T04:31:35.061294+0000","last_fullsized":"2023-10-06T04:31:35.061294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:59.355740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.030228+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:31:34.030228+0000","last_peered":"2023-10-06T04:31:34.030228+0000","last_clean":"2023-10-06T04:31:34.030228+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:31:34.030228+0000","last_undegraded":"2023-10-06T04:31:34.030228+0000","last_fullsized":"2023-10-06T04:31:34.030228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.047752+0000","last_change":"2023-10-06T04:31:34.043758+0000","last_active":"2023-10-06T04:31:34.047752+0000","last_peered":"2023-10-06T04:31:34.047752+0000","last_clean":"2023-10-06T04:31:34.047752+0000","last_became_active":"2023-10-06T04:31:34.043386+0000","last_became_peered":"2023-10-06T04:31:34.043386+0000","last_unstale":"2023-10-06T04:31:34.047752+0000","last_undegraded":"2023-10-06T04:31:34.047752+0000","last_fullsized":"2023-10-06T04:31:34.047752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058248+0000","last_change":"2023-10-06T04:31:35.058248+0000","last_active":"2023-10-06T04:31:35.058248+0000","last_peered":"2023-10-06T04:31:35.058248+0000","last_clean":"2023-10-06T04:31:35.058248+0000","last_became_active":"2023-10-06T04:31:35.057955+0000","last_became_peered":"2023-10-06T04:31:35.057955+0000","last_unstale":"2023-10-06T04:31:35.058248+0000","last_undegraded":"2023-10-06T04:31:35.058248+0000","last_fullsized":"2023-10-06T04:31:35.058248+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.027787+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:31:34.027787+0000","last_peered":"2023-10-06T04:31:34.027787+0000","last_clean":"2023-10-06T04:31:34.027787+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:31:34.027787+0000","last_undegraded":"2023-10-06T04:31:34.027787+0000","last_fullsized":"2023-10-06T04:31:34.027787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.044046+0000","last_change":"2023-10-06T04:31:34.041900+0000","last_active":"2023-10-06T04:31:34.044046+0000","last_peered":"2023-10-06T04:31:34.044046+0000","last_clean":"2023-10-06T04:31:34.044046+0000","last_became_active":"2023-10-06T04:31:34.041621+0000","last_became_peered":"2023-10-06T04:31:34.041621+0000","last_unstale":"2023-10-06T04:31:34.044046+0000","last_undegraded":"2023-10-06T04:31:34.044046+0000","last_fullsized":"2023-10-06T04:31:34.044046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060634+0000","last_change":"2023-10-06T04:31:35.060634+0000","last_active":"2023-10-06T04:31:35.060634+0000","last_peered":"2023-10-06T04:31:35.060634+0000","last_clean":"2023-10-06T04:31:35.060634+0000","last_became_active":"2023-10-06T04:31:35.060183+0000","last_became_peered":"2023-10-06T04:31:35.060183+0000","last_unstale":"2023-10-06T04:31:35.060634+0000","last_undegraded":"2023-10-06T04:31:35.060634+0000","last_fullsized":"2023-10-06T04:31:35.060634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.030112+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:31:34.030112+0000","last_peered":"2023-10-06T04:31:34.030112+0000","last_clean":"2023-10-06T04:31:34.030112+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:31:34.030112+0000","last_undegraded":"2023-10-06T04:31:34.030112+0000","last_fullsized":"2023-10-06T04:31:34.030112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.036078+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:31:34.036078+0000","last_peered":"2023-10-06T04:31:34.036078+0000","last_clean":"2023-10-06T04:31:34.036078+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:31:34.036078+0000","last_undegraded":"2023-10-06T04:31:34.036078+0000","last_fullsized":"2023-10-06T04:31:34.036078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.044001+0000","last_change":"2023-10-06T04:31:34.039806+0000","last_active":"2023-10-06T04:31:34.044001+0000","last_peered":"2023-10-06T04:31:34.044001+0000","last_clean":"2023-10-06T04:31:34.044001+0000","last_became_active":"2023-10-06T04:31:34.039496+0000","last_became_peered":"2023-10-06T04:31:34.039496+0000","last_unstale":"2023-10-06T04:31:34.044001+0000","last_undegraded":"2023-10-06T04:31:34.044001+0000","last_fullsized":"2023-10-06T04:31:34.044001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050727+0000","last_change":"2023-10-06T04:31:35.050727+0000","last_active":"2023-10-06T04:31:35.050727+0000","last_peered":"2023-10-06T04:31:35.050727+0000","last_clean":"2023-10-06T04:31:35.050727+0000","last_became_active":"2023-10-06T04:31:35.050446+0000","last_became_peered":"2023-10-06T04:31:35.050446+0000","last_unstale":"2023-10-06T04:31:35.050727+0000","last_undegraded":"2023-10-06T04:31:35.050727+0000","last_fullsized":"2023-10-06T04:31:35.050727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:08.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.036161+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:31:34.036161+0000","last_peered":"2023-10-06T04:31:34.036161+0000","last_clean":"2023-10-06T04:31:34.036161+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:31:34.036161+0000","last_undegraded":"2023-10-06T04:31:34.036161+0000","last_fullsized":"2023-10-06T04:31:34.036161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.042513+0000","last_change":"2023-10-06T04:31:34.039113+0000","last_active":"2023-10-06T04:31:34.042513+0000","last_peered":"2023-10-06T04:31:34.042513+0000","last_clean":"2023-10-06T04:31:34.042513+0000","last_became_active":"2023-10-06T04:31:34.038410+0000","last_became_peered":"2023-10-06T04:31:34.038410+0000","last_unstale":"2023-10-06T04:31:34.042513+0000","last_undegraded":"2023-10-06T04:31:34.042513+0000","last_fullsized":"2023-10-06T04:31:34.042513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059901+0000","last_change":"2023-10-06T04:31:35.059901+0000","last_active":"2023-10-06T04:31:35.059901+0000","last_peered":"2023-10-06T04:31:35.059901+0000","last_clean":"2023-10-06T04:31:35.059901+0000","last_became_active":"2023-10-06T04:31:35.059461+0000","last_became_peered":"2023-10-06T04:31:35.059461+0000","last_unstale":"2023-10-06T04:31:35.059901+0000","last_undegraded":"2023-10-06T04:31:35.059901+0000","last_fullsized":"2023-10-06T04:31:35.059901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.037722+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:31:34.037722+0000","last_peered":"2023-10-06T04:31:34.037722+0000","last_clean":"2023-10-06T04:31:34.037722+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:31:34.037722+0000","last_undegraded":"2023-10-06T04:31:34.037722+0000","last_fullsized":"2023-10-06T04:31:34.037722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.044169+0000","last_change":"2023-10-06T04:31:34.042477+0000","last_active":"2023-10-06T04:31:34.044169+0000","last_peered":"2023-10-06T04:31:34.044169+0000","last_clean":"2023-10-06T04:31:34.044169+0000","last_became_active":"2023-10-06T04:31:34.041582+0000","last_became_peered":"2023-10-06T04:31:34.041582+0000","last_unstale":"2023-10-06T04:31:34.044169+0000","last_undegraded":"2023-10-06T04:31:34.044169+0000","last_fullsized":"2023-10-06T04:31:34.044169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061904+0000","last_change":"2023-10-06T04:31:35.061904+0000","last_active":"2023-10-06T04:31:35.061904+0000","last_peered":"2023-10-06T04:31:35.061904+0000","last_clean":"2023-10-06T04:31:35.061904+0000","last_became_active":"2023-10-06T04:31:35.061655+0000","last_became_peered":"2023-10-06T04:31:35.061655+0000","last_unstale":"2023-10-06T04:31:35.061904+0000","last_undegraded":"2023-10-06T04:31:35.061904+0000","last_fullsized":"2023-10-06T04:31:35.061904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255152+0000","last_change":"2023-10-06T04:31:34.042617+0000","last_active":"2023-10-06T04:31:35.255152+0000","last_peered":"2023-10-06T04:31:35.255152+0000","last_clean":"2023-10-06T04:31:35.255152+0000","last_became_active":"2023-10-06T04:31:34.042346+0000","last_became_peered":"2023-10-06T04:31:34.042346+0000","last_unstale":"2023-10-06T04:31:35.255152+0000","last_undegraded":"2023-10-06T04:31:35.255152+0000","last_fullsized":"2023-10-06T04:31:35.255152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.037532+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:31:34.037532+0000","last_peered":"2023-10-06T04:31:34.037532+0000","last_clean":"2023-10-06T04:31:34.037532+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:31:34.037532+0000","last_undegraded":"2023-10-06T04:31:34.037532+0000","last_fullsized":"2023-10-06T04:31:34.037532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060668+0000","last_change":"2023-10-06T04:31:35.060668+0000","last_active":"2023-10-06T04:31:35.060668+0000","last_peered":"2023-10-06T04:31:35.060668+0000","last_clean":"2023-10-06T04:31:35.060668+0000","last_became_active":"2023-10-06T04:31:35.059927+0000","last_became_peered":"2023-10-06T04:31:35.059927+0000","last_unstale":"2023-10-06T04:31:35.060668+0000","last_undegraded":"2023-10-06T04:31:35.060668+0000","last_fullsized":"2023-10-06T04:31:35.060668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.030721+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:31:34.030721+0000","last_peered":"2023-10-06T04:31:34.030721+0000","last_clean":"2023-10-06T04:31:34.030721+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:31:34.030721+0000","last_undegraded":"2023-10-06T04:31:34.030721+0000","last_fullsized":"2023-10-06T04:31:34.030721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051614+0000","last_change":"2023-10-06T04:31:34.044177+0000","last_active":"2023-10-06T04:31:34.051614+0000","last_peered":"2023-10-06T04:31:34.051614+0000","last_clean":"2023-10-06T04:31:34.051614+0000","last_became_active":"2023-10-06T04:31:34.043921+0000","last_became_peered":"2023-10-06T04:31:34.043921+0000","last_unstale":"2023-10-06T04:31:34.051614+0000","last_undegraded":"2023-10-06T04:31:34.051614+0000","last_fullsized":"2023-10-06T04:31:34.051614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059320+0000","last_change":"2023-10-06T04:31:35.059320+0000","last_active":"2023-10-06T04:31:35.059320+0000","last_peered":"2023-10-06T04:31:35.059320+0000","last_clean":"2023-10-06T04:31:35.059320+0000","last_became_active":"2023-10-06T04:31:35.058458+0000","last_became_peered":"2023-10-06T04:31:35.058458+0000","last_unstale":"2023-10-06T04:31:35.059320+0000","last_undegraded":"2023-10-06T04:31:35.059320+0000","last_fullsized":"2023-10-06T04:31:35.059320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:06.614824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.045324+0000","last_change":"2023-10-06T04:31:34.043203+0000","last_active":"2023-10-06T04:31:35.045324+0000","last_peered":"2023-10-06T04:31:35.045324+0000","last_clean":"2023-10-06T04:31:35.045324+0000","last_became_active":"2023-10-06T04:31:34.042886+0000","last_became_peered":"2023-10-06T04:31:34.042886+0000","last_unstale":"2023-10-06T04:31:35.045324+0000","last_undegraded":"2023-10-06T04:31:35.045324+0000","last_fullsized":"2023-10-06T04:31:35.045324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057021+0000","last_change":"2023-10-06T04:31:35.057021+0000","last_active":"2023-10-06T04:31:35.057021+0000","last_peered":"2023-10-06T04:31:35.057021+0000","last_clean":"2023-10-06T04:31:35.057021+0000","last_became_active":"2023-10-06T04:31:35.056770+0000","last_became_peered":"2023-10-06T04:31:35.056770+0000","last_unstale":"2023-10-06T04:31:35.057021+0000","last_undegraded":"2023-10-06T04:31:35.057021+0000","last_fullsized":"2023-10-06T04:31:35.057021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255029+0000","last_change":"2023-10-06T04:31:34.044080+0000","last_active":"2023-10-06T04:31:35.255029+0000","last_peered":"2023-10-06T04:31:35.255029+0000","last_clean":"2023-10-06T04:31:35.255029+0000","last_became_active":"2023-10-06T04:31:34.042225+0000","last_became_peered":"2023-10-06T04:31:34.042225+0000","last_unstale":"2023-10-06T04:31:35.255029+0000","last_undegraded":"2023-10-06T04:31:35.255029+0000","last_fullsized":"2023-10-06T04:31:35.255029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:17.059659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055021+0000","last_change":"2023-10-06T04:31:35.055021+0000","last_active":"2023-10-06T04:31:35.055021+0000","last_peered":"2023-10-06T04:31:35.055021+0000","last_clean":"2023-10-06T04:31:35.055021+0000","last_became_active":"2023-10-06T04:31:35.054737+0000","last_became_peered":"2023-10-06T04:31:35.054737+0000","last_unstale":"2023-10-06T04:31:35.055021+0000","last_undegraded":"2023-10-06T04:31:35.055021+0000","last_fullsized":"2023-10-06T04:31:35.055021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:26.087138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257207+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:35.257207+0000","last_peered":"2023-10-06T04:31:35.257207+0000","last_clean":"2023-10-06T04:31:35.257207+0000","last_became_active":"2023-10-06T04:31:34.041996+0000","last_became_peered":"2023-10-06T04:31:34.041996+0000","last_unstale":"2023-10-06T04:31:35.257207+0000","last_undegraded":"2023-10-06T04:31:35.257207+0000","last_fullsized":"2023-10-06T04:31:35.257207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059718+0000","last_change":"2023-10-06T04:31:35.059718+0000","last_active":"2023-10-06T04:31:35.059718+0000","last_peered":"2023-10-06T04:31:35.059718+0000","last_clean":"2023-10-06T04:31:35.059718+0000","last_became_active":"2023-10-06T04:31:35.059438+0000","last_became_peered":"2023-10-06T04:31:35.059438+0000","last_unstale":"2023-10-06T04:31:35.059718+0000","last_undegraded":"2023-10-06T04:31:35.059718+0000","last_fullsized":"2023-10-06T04:31:35.059718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.047916+0000","last_change":"2023-10-06T04:31:34.047689+0000","last_active":"2023-10-06T04:31:35.047916+0000","last_peered":"2023-10-06T04:31:35.047916+0000","last_clean":"2023-10-06T04:31:35.047916+0000","last_became_active":"2023-10-06T04:31:34.047311+0000","last_became_peered":"2023-10-06T04:31:34.047311+0000","last_unstale":"2023-10-06T04:31:35.047916+0000","last_undegraded":"2023-10-06T04:31:35.047916+0000","last_fullsized":"2023-10-06T04:31:35.047916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055295+0000","last_change":"2023-10-06T04:31:35.055295+0000","last_active":"2023-10-06T04:31:35.055295+0000","last_peered":"2023-10-06T04:31:35.055295+0000","last_clean":"2023-10-06T04:31:35.055295+0000","last_became_active":"2023-10-06T04:31:35.055019+0000","last_became_peered":"2023-10-06T04:31:35.055019+0000","last_unstale":"2023-10-06T04:31:35.055295+0000","last_undegraded":"2023-10-06T04:31:35.055295+0000","last_fullsized":"2023-10-06T04:31:35.055295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257565+0000","last_change":"2023-10-06T04:31:34.040784+0000","last_active":"2023-10-06T04:31:35.257565+0000","last_peered":"2023-10-06T04:31:35.257565+0000","last_clean":"2023-10-06T04:31:35.257565+0000","last_became_active":"2023-10-06T04:31:34.040127+0000","last_became_peered":"2023-10-06T04:31:34.040127+0000","last_unstale":"2023-10-06T04:31:35.257565+0000","last_undegraded":"2023-10-06T04:31:35.257565+0000","last_fullsized":"2023-10-06T04:31:35.257565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052827+0000","last_change":"2023-10-06T04:31:35.052827+0000","last_active":"2023-10-06T04:31:35.052827+0000","last_peered":"2023-10-06T04:31:35.052827+0000","last_clean":"2023-10-06T04:31:35.052827+0000","last_became_active":"2023-10-06T04:31:35.052539+0000","last_became_peered":"2023-10-06T04:31:35.052539+0000","last_unstale":"2023-10-06T04:31:35.052827+0000","last_undegraded":"2023-10-06T04:31:35.052827+0000","last_fullsized":"2023-10-06T04:31:35.052827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051004+0000","last_change":"2023-10-06T04:31:35.051004+0000","last_active":"2023-10-06T04:31:35.051004+0000","last_peered":"2023-10-06T04:31:35.051004+0000","last_clean":"2023-10-06T04:31:35.051004+0000","last_became_active":"2023-10-06T04:31:35.050694+0000","last_became_peered":"2023-10-06T04:31:35.050694+0000","last_unstale":"2023-10-06T04:31:35.051004+0000","last_undegraded":"2023-10-06T04:31:35.051004+0000","last_fullsized":"2023-10-06T04:31:35.051004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062748+0000","last_change":"2023-10-06T04:31:35.062748+0000","last_active":"2023-10-06T04:31:35.062748+0000","last_peered":"2023-10-06T04:31:35.062748+0000","last_clean":"2023-10-06T04:31:35.062748+0000","last_became_active":"2023-10-06T04:31:35.060650+0000","last_became_peered":"2023-10-06T04:31:35.060650+0000","last_unstale":"2023-10-06T04:31:35.062748+0000","last_undegraded":"2023-10-06T04:31:35.062748+0000","last_fullsized":"2023-10-06T04:31:35.062748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052675+0000","last_change":"2023-10-06T04:31:35.052675+0000","last_active":"2023-10-06T04:31:35.052675+0000","last_peered":"2023-10-06T04:31:35.052675+0000","last_clean":"2023-10-06T04:31:35.052675+0000","last_became_active":"2023-10-06T04:31:35.052379+0000","last_became_peered":"2023-10-06T04:31:35.052379+0000","last_unstale":"2023-10-06T04:31:35.052675+0000","last_undegraded":"2023-10-06T04:31:35.052675+0000","last_fullsized":"2023-10-06T04:31:35.052675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:40.270087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061097+0000","last_change":"2023-10-06T04:31:35.061097+0000","last_active":"2023-10-06T04:31:35.061097+0000","last_peered":"2023-10-06T04:31:35.061097+0000","last_clean":"2023-10-06T04:31:35.061097+0000","last_became_active":"2023-10-06T04:31:35.060817+0000","last_became_peered":"2023-10-06T04:31:35.060817+0000","last_unstale":"2023-10-06T04:31:35.061097+0000","last_undegraded":"2023-10-06T04:31:35.061097+0000","last_fullsized":"2023-10-06T04:31:35.061097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062003+0000","last_change":"2023-10-06T04:31:35.062003+0000","last_active":"2023-10-06T04:31:35.062003+0000","last_peered":"2023-10-06T04:31:35.062003+0000","last_clean":"2023-10-06T04:31:35.062003+0000","last_became_active":"2023-10-06T04:31:35.061553+0000","last_became_peered":"2023-10-06T04:31:35.061553+0000","last_unstale":"2023-10-06T04:31:35.062003+0000","last_undegraded":"2023-10-06T04:31:35.062003+0000","last_fullsized":"2023-10-06T04:31:35.062003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":673906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":673046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:31:39.310 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:31:39.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.459+0000 7f84dd386700 1 Processor -- start 2023-10-06T04:31:39.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.459+0000 7f84dd386700 1 -- start start 2023-10-06T04:31:39.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84dd386700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f84d80ac720 0x7f84d80ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84dd386700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84d806a750 0x7f84d806abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84dd386700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84d80aeea0 con 0x7f84d80680b0 2023-10-06T04:31:39.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84dd386700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d806deb0 con 0x7f84d806a750 2023-10-06T04:31:39.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84dd386700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84d80aecc0 con 0x7f84d80ac720 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84d77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f84d806a750 0x7f84d806abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84d67fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f84d80680b0 0x7f84d80acaf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33306/0 (socket says 172.21.15.123:33306) 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.460+0000 7f84d67fc700 1 -- 172.21.15.123:0/4290171125 learned_addr learned my addr 172.21.15.123:0/4290171125 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d6ffd700 1 --2- 172.21.15.123:0/4290171125 >> v2:172.21.15.177:3300/0 conn(0x7f84d80ac720 0x7f84d80ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d77fe700 1 -- 172.21.15.123:0/4290171125 >> v2:172.21.15.177:3300/0 conn(0x7f84d80ac720 msgr2=0x7f84d80ae780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d77fe700 1 --2- 172.21.15.123:0/4290171125 >> v2:172.21.15.177:3300/0 conn(0x7f84d80ac720 0x7f84d80ae780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d77fe700 1 -- 172.21.15.123:0/4290171125 >> v1:172.21.15.123:6789/0 conn(0x7f84d80680b0 legacy=0x7f84d80acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d77fe700 1 -- 172.21.15.123:0/4290171125 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d80af180 con 0x7f84d806a750 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d77fe700 1 --2- 172.21.15.123:0/4290171125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d806abb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f84d80c99c0 tx=0x7f84d0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51915be9f60ea9e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d5ffb700 1 -- 172.21.15.123:0/4290171125 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d000e070 con 0x7f84d806a750 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d5ffb700 1 -- 172.21.15.123:0/4290171125 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84d000bc80 con 0x7f84d806a750 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84d5ffb700 1 -- 172.21.15.123:0/4290171125 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84d0012430 con 0x7f84d806a750 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84dd386700 1 -- 172.21.15.123:0/4290171125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 msgr2=0x7f84d806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84dd386700 1 --2- 172.21.15.123:0/4290171125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d806abb0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f84d80c99c0 tx=0x7f84d0009780 comp rx=0 tx=0).stop 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84dd386700 1 -- 172.21.15.123:0/4290171125 shutdown_connections 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84dd386700 1 --2- 172.21.15.123:0/4290171125 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d806abb0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84dd386700 1 --2- 172.21.15.123:0/4290171125 >> v2:172.21.15.177:3300/0 conn(0x7f84d80ac720 0x7f84d80ae780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84dd386700 1 -- 172.21.15.123:0/4290171125 >> 172.21.15.123:0/4290171125 conn(0x7f84d80cda80 msgr2=0x7f84d80c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.461+0000 7f84dd386700 1 -- 172.21.15.123:0/4290171125 shutdown_connections 2023-10-06T04:31:39.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.462+0000 7f84dd386700 1 -- 172.21.15.123:0/4290171125 wait complete. 2023-10-06T04:31:39.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 Processor -- start 2023-10-06T04:31:39.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 -- start start 2023-10-06T04:31:39.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80680b0 0x7f84d80b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d8146400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80ac720 0x7f84d806ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80c2260 con 0x7f84d806a750 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80b0ca0 con 0x7f84d80680b0 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.463+0000 7f84dd386700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84d80b1130 con 0x7f84d80ac720 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d8146400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d8146400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52298/0 (socket says 172.21.15.123:52298) 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 -- 172.21.15.123:0/1811130454 learned_addr learned my addr 172.21.15.123:0/1811130454 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d6ffd700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80680b0 0x7f84d80b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d77fe700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80ac720 0x7f84d806ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 -- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80ac720 msgr2=0x7f84d806ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80ac720 0x7f84d806ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 -- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80680b0 msgr2=0x7f84d80b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80680b0 0x7f84d80b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 -- 172.21.15.123:0/1811130454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84d0009a20 con 0x7f84d806a750 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84d67fc700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d8146400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c7f0 tx=0x7f84c8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84ceffd700 1 -- 172.21.15.123:0/1811130454 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8005b30 con 0x7f84d806a750 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84ceffd700 1 -- 172.21.15.123:0/1811130454 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84c8005cb0 con 0x7f84d806a750 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84d8146940 con 0x7f84d806a750 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84ceffd700 1 -- 172.21.15.123:0/1811130454 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84c8004a80 con 0x7f84d806a750 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.464+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84d816a6f0 con 0x7f84d806a750 2023-10-06T04:31:39.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.465+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84d80c2410 con 0x7f84d806a750 2023-10-06T04:31:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.465+0000 7f84ceffd700 1 -- 172.21.15.123:0/1811130454 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f84c80070d0 con 0x7f84d806a750 2023-10-06T04:31:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.466+0000 7f84ceffd700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f84b8066f30 0x7f84b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.466+0000 7f84ceffd700 1 -- 172.21.15.123:0/1811130454 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f84c8083960 con 0x7f84d806a750 2023-10-06T04:31:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.466+0000 7f84d6ffd700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f84b8066f30 0x7f84b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.466+0000 7f84d6ffd700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f84b8066f30 0x7f84b80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006fd0 tx=0x7f84c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.471 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.470+0000 7f84ceffd700 1 -- 172.21.15.123:0/1811130454 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f84c804b5a0 con 0x7f84d806a750 2023-10-06T04:31:39.648 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.646+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f84d80cde80 con 0x7f84b8066f30 2023-10-06T04:31:39.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.654+0000 7f84ceffd700 1 -- 172.21.15.123:0/1811130454 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286715 (secure 0 0 0) 0x7f84d80cde80 con 0x7f84b8066f30 2023-10-06T04:31:39.659 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f84b8066f30 msgr2=0x7f84b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f84b8066f30 0x7f84b80693e0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f84c0006fd0 tx=0x7f84c0008040 comp rx=0 tx=0).stop 2023-10-06T04:31:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 msgr2=0x7f84d8146400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d8146400 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f84c800c7f0 tx=0x7f84c8007f70 comp rx=0 tx=0).stop 2023-10-06T04:31:39.661 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 shutdown_connections 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f84b8066f30 0x7f84b80693e0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f84d80ac720 0x7f84d806ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f84d80680b0 0x7f84d80b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 --2- 172.21.15.123:0/1811130454 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f84d806a750 0x7f84d8146400 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 >> 172.21.15.123:0/1811130454 conn(0x7f84d80cda80 msgr2=0x7f84d80cc880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 shutdown_connections 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.657+0000 7f84dd386700 1 -- 172.21.15.123:0/1811130454 wait complete. 2023-10-06T04:31:39.662 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:31:39.670 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-10-06T04:31:38.882081+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":76,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381280,"kb_used_data":2976,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593440,"statfs":{"total":773094113280,"available":770655682560,"internally_reserved":0,"allocated":3047424,"data_stored":1843020,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.859937"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057310+0000","last_change":"2023-10-06T04:31:35.057310+0000","last_active":"2023-10-06T04:31:35.057310+0000","last_peered":"2023-10-06T04:31:35.057310+0000","last_clean":"2023-10-06T04:31:35.057310+0000","last_became_active":"2023-10-06T04:31:35.057042+0000","last_became_peered":"2023-10-06T04:31:35.057042+0000","last_unstale":"2023-10-06T04:31:35.057310+0000","last_undegraded":"2023-10-06T04:31:35.057310+0000","last_fullsized":"2023-10-06T04:31:35.057310+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063658+0000","last_change":"2023-10-06T04:31:35.063658+0000","last_active":"2023-10-06T04:31:35.063658+0000","last_peered":"2023-10-06T04:31:35.063658+0000","last_clean":"2023-10-06T04:31:35.063658+0000","last_became_active":"2023-10-06T04:31:35.063403+0000","last_became_peered":"2023-10-06T04:31:35.063403+0000","last_unstale":"2023-10-06T04:31:35.063658+0000","last_undegraded":"2023-10-06T04:31:35.063658+0000","last_fullsized":"2023-10-06T04:31:35.063658+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.056211+0000","last_change":"2023-10-06T04:31:35.056211+0000","last_active":"2023-10-06T04:31:35.056211+0000","last_peered":"2023-10-06T04:31:35.056211+0000","last_clean":"2023-10-06T04:31:35.056211+0000","last_became_active":"2023-10-06T04:31:35.055883+0000","last_became_peered":"2023-10-06T04:31:35.055883+0000","last_unstale":"2023-10-06T04:31:35.056211+0000","last_undegraded":"2023-10-06T04:31:35.056211+0000","last_fullsized":"2023-10-06T04:31:35.056211+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:54.893111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055601+0000","last_change":"2023-10-06T04:31:35.055601+0000","last_active":"2023-10-06T04:31:35.055601+0000","last_peered":"2023-10-06T04:31:35.055601+0000","last_clean":"2023-10-06T04:31:35.055601+0000","last_became_active":"2023-10-06T04:31:35.055325+0000","last_became_peered":"2023-10-06T04:31:35.055325+0000","last_unstale":"2023-10-06T04:31:35.055601+0000","last_undegraded":"2023-10-06T04:31:35.055601+0000","last_fullsized":"2023-10-06T04:31:35.055601+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061888+0000","last_change":"2023-10-06T04:31:35.061888+0000","last_active":"2023-10-06T04:31:35.061888+0000","last_peered":"2023-10-06T04:31:35.061888+0000","last_clean":"2023-10-06T04:31:35.061888+0000","last_became_active":"2023-10-06T04:31:35.061635+0000","last_became_peered":"2023-10-06T04:31:35.061635+0000","last_unstale":"2023-10-06T04:31:35.061888+0000","last_undegraded":"2023-10-06T04:31:35.061888+0000","last_fullsized":"2023-10-06T04:31:35.061888+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055786+0000","last_change":"2023-10-06T04:31:35.055786+0000","last_active":"2023-10-06T04:31:35.055786+0000","last_peered":"2023-10-06T04:31:35.055786+0000","last_clean":"2023-10-06T04:31:35.055786+0000","last_became_active":"2023-10-06T04:31:35.055501+0000","last_became_peered":"2023-10-06T04:31:35.055501+0000","last_unstale":"2023-10-06T04:31:35.055786+0000","last_undegraded":"2023-10-06T04:31:35.055786+0000","last_fullsized":"2023-10-06T04:31:35.055786+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:47.521895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065611+0000","last_change":"2023-10-06T04:31:35.065611+0000","last_active":"2023-10-06T04:31:35.065611+0000","last_peered":"2023-10-06T04:31:35.065611+0000","last_clean":"2023-10-06T04:31:35.065611+0000","last_became_active":"2023-10-06T04:31:35.064864+0000","last_became_peered":"2023-10-06T04:31:35.064864+0000","last_unstale":"2023-10-06T04:31:35.065611+0000","last_undegraded":"2023-10-06T04:31:35.065611+0000","last_fullsized":"2023-10-06T04:31:35.065611+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060662+0000","last_change":"2023-10-06T04:31:35.060662+0000","last_active":"2023-10-06T04:31:35.060662+0000","last_peered":"2023-10-06T04:31:35.060662+0000","last_clean":"2023-10-06T04:31:35.060662+0000","last_became_active":"2023-10-06T04:31:35.060371+0000","last_became_peered":"2023-10-06T04:31:35.060371+0000","last_unstale":"2023-10-06T04:31:35.060662+0000","last_undegraded":"2023-10-06T04:31:35.060662+0000","last_fullsized":"2023-10-06T04:31:35.060662+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061068+0000","last_change":"2023-10-06T04:31:35.061068+0000","last_active":"2023-10-06T04:31:35.061068+0000","last_peered":"2023-10-06T04:31:35.061068+0000","last_clean":"2023-10-06T04:31:35.061068+0000","last_became_active":"2023-10-06T04:31:35.060663+0000","last_became_peered":"2023-10-06T04:31:35.060663+0000","last_unstale":"2023-10-06T04:31:35.061068+0000","last_undegraded":"2023-10-06T04:31:35.061068+0000","last_fullsized":"2023-10-06T04:31:35.061068+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062498+0000","last_change":"2023-10-06T04:31:35.062498+0000","last_active":"2023-10-06T04:31:35.062498+0000","last_peered":"2023-10-06T04:31:35.062498+0000","last_clean":"2023-10-06T04:31:35.062498+0000","last_became_active":"2023-10-06T04:31:35.062236+0000","last_became_peered":"2023-10-06T04:31:35.062236+0000","last_unstale":"2023-10-06T04:31:35.062498+0000","last_undegraded":"2023-10-06T04:31:35.062498+0000","last_fullsized":"2023-10-06T04:31:35.062498+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057945+0000","last_change":"2023-10-06T04:31:35.057945+0000","last_active":"2023-10-06T04:31:35.057945+0000","last_peered":"2023-10-06T04:31:35.057945+0000","last_clean":"2023-10-06T04:31:35.057945+0000","last_became_active":"2023-10-06T04:31:35.057568+0000","last_became_peered":"2023-10-06T04:31:35.057568+0000","last_unstale":"2023-10-06T04:31:35.057945+0000","last_undegraded":"2023-10-06T04:31:35.057945+0000","last_fullsized":"2023-10-06T04:31:35.057945+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:55.588795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051600+0000","last_change":"2023-10-06T04:31:35.051600+0000","last_active":"2023-10-06T04:31:35.051600+0000","last_peered":"2023-10-06T04:31:35.051600+0000","last_clean":"2023-10-06T04:31:35.051600+0000","last_became_active":"2023-10-06T04:31:35.051344+0000","last_became_peered":"2023-10-06T04:31:35.051344+0000","last_unstale":"2023-10-06T04:31:35.051600+0000","last_undegraded":"2023-10-06T04:31:35.051600+0000","last_fullsized":"2023-10-06T04:31:35.051600+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055073+0000","last_change":"2023-10-06T04:31:35.055073+0000","last_active":"2023-10-06T04:31:35.055073+0000","last_peered":"2023-10-06T04:31:35.055073+0000","last_clean":"2023-10-06T04:31:35.055073+0000","last_became_active":"2023-10-06T04:31:35.054723+0000","last_became_peered":"2023-10-06T04:31:35.054723+0000","last_unstale":"2023-10-06T04:31:35.055073+0000","last_undegraded":"2023-10-06T04:31:35.055073+0000","last_fullsized":"2023-10-06T04:31:35.055073+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:53.494026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065191+0000","last_change":"2023-10-06T04:31:35.065191+0000","last_active":"2023-10-06T04:31:35.065191+0000","last_peered":"2023-10-06T04:31:35.065191+0000","last_clean":"2023-10-06T04:31:35.065191+0000","last_became_active":"2023-10-06T04:31:35.064865+0000","last_became_peered":"2023-10-06T04:31:35.064865+0000","last_unstale":"2023-10-06T04:31:35.065191+0000","last_undegraded":"2023-10-06T04:31:35.065191+0000","last_fullsized":"2023-10-06T04:31:35.065191+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065835+0000","last_change":"2023-10-06T04:31:35.065835+0000","last_active":"2023-10-06T04:31:35.065835+0000","last_peered":"2023-10-06T04:31:35.065835+0000","last_clean":"2023-10-06T04:31:35.065835+0000","last_became_active":"2023-10-06T04:31:35.065225+0000","last_became_peered":"2023-10-06T04:31:35.065225+0000","last_unstale":"2023-10-06T04:31:35.065835+0000","last_undegraded":"2023-10-06T04:31:35.065835+0000","last_fullsized":"2023-10-06T04:31:35.065835+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050260+0000","last_change":"2023-10-06T04:31:35.050260+0000","last_active":"2023-10-06T04:31:35.050260+0000","last_peered":"2023-10-06T04:31:35.050260+0000","last_clean":"2023-10-06T04:31:35.050260+0000","last_became_active":"2023-10-06T04:31:35.049943+0000","last_became_peered":"2023-10-06T04:31:35.049943+0000","last_unstale":"2023-10-06T04:31:35.050260+0000","last_undegraded":"2023-10-06T04:31:35.050260+0000","last_fullsized":"2023-10-06T04:31:35.050260+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060912+0000","last_change":"2023-10-06T04:31:35.060912+0000","last_active":"2023-10-06T04:31:35.060912+0000","last_peered":"2023-10-06T04:31:35.060912+0000","last_clean":"2023-10-06T04:31:35.060912+0000","last_became_active":"2023-10-06T04:31:35.060330+0000","last_became_peered":"2023-10-06T04:31:35.060330+0000","last_unstale":"2023-10-06T04:31:35.060912+0000","last_undegraded":"2023-10-06T04:31:35.060912+0000","last_fullsized":"2023-10-06T04:31:35.060912+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057646+0000","last_change":"2023-10-06T04:31:35.057646+0000","last_active":"2023-10-06T04:31:35.057646+0000","last_peered":"2023-10-06T04:31:35.057646+0000","last_clean":"2023-10-06T04:31:35.057646+0000","last_became_active":"2023-10-06T04:31:35.057375+0000","last_became_peered":"2023-10-06T04:31:35.057375+0000","last_unstale":"2023-10-06T04:31:35.057646+0000","last_undegraded":"2023-10-06T04:31:35.057646+0000","last_fullsized":"2023-10-06T04:31:35.057646+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:44.450170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046638+0000","last_change":"2023-10-06T04:31:35.046638+0000","last_active":"2023-10-06T04:31:35.046638+0000","last_peered":"2023-10-06T04:31:35.046638+0000","last_clean":"2023-10-06T04:31:35.046638+0000","last_became_active":"2023-10-06T04:31:35.046376+0000","last_became_peered":"2023-10-06T04:31:35.046376+0000","last_unstale":"2023-10-06T04:31:35.046638+0000","last_undegraded":"2023-10-06T04:31:35.046638+0000","last_fullsized":"2023-10-06T04:31:35.046638+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:33.326007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052597+0000","last_change":"2023-10-06T04:31:35.052597+0000","last_active":"2023-10-06T04:31:35.052597+0000","last_peered":"2023-10-06T04:31:35.052597+0000","last_clean":"2023-10-06T04:31:35.052597+0000","last_became_active":"2023-10-06T04:31:35.052318+0000","last_became_peered":"2023-10-06T04:31:35.052318+0000","last_unstale":"2023-10-06T04:31:35.052597+0000","last_undegraded":"2023-10-06T04:31:35.052597+0000","last_fullsized":"2023-10-06T04:31:35.052597+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.067001+0000","last_change":"2023-10-06T04:31:35.067001+0000","last_active":"2023-10-06T04:31:35.067001+0000","last_peered":"2023-10-06T04:31:35.067001+0000","last_clean":"2023-10-06T04:31:35.067001+0000","last_became_active":"2023-10-06T04:31:35.066763+0000","last_became_peered":"2023-10-06T04:31:35.066763+0000","last_unstale":"2023-10-06T04:31:35.067001+0000","last_undegraded":"2023-10-06T04:31:35.067001+0000","last_fullsized":"2023-10-06T04:31:35.067001+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059383+0000","last_change":"2023-10-06T04:31:35.059383+0000","last_active":"2023-10-06T04:31:35.059383+0000","last_peered":"2023-10-06T04:31:35.059383+0000","last_clean":"2023-10-06T04:31:35.059383+0000","last_became_active":"2023-10-06T04:31:35.059106+0000","last_became_peered":"2023-10-06T04:31:35.059106+0000","last_unstale":"2023-10-06T04:31:35.059383+0000","last_undegraded":"2023-10-06T04:31:35.059383+0000","last_fullsized":"2023-10-06T04:31:35.059383+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054733+0000","last_change":"2023-10-06T04:31:35.054733+0000","last_active":"2023-10-06T04:31:35.054733+0000","last_peered":"2023-10-06T04:31:35.054733+0000","last_clean":"2023-10-06T04:31:35.054733+0000","last_became_active":"2023-10-06T04:31:35.054454+0000","last_became_peered":"2023-10-06T04:31:35.054454+0000","last_unstale":"2023-10-06T04:31:35.054733+0000","last_undegraded":"2023-10-06T04:31:35.054733+0000","last_fullsized":"2023-10-06T04:31:35.054733+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059648+0000","last_change":"2023-10-06T04:31:35.059648+0000","last_active":"2023-10-06T04:31:35.059648+0000","last_peered":"2023-10-06T04:31:35.059648+0000","last_clean":"2023-10-06T04:31:35.059648+0000","last_became_active":"2023-10-06T04:31:35.059380+0000","last_became_peered":"2023-10-06T04:31:35.059380+0000","last_unstale":"2023-10-06T04:31:35.059648+0000","last_undegraded":"2023-10-06T04:31:35.059648+0000","last_fullsized":"2023-10-06T04:31:35.059648+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063285+0000","last_change":"2023-10-06T04:31:35.063285+0000","last_active":"2023-10-06T04:31:35.063285+0000","last_peered":"2023-10-06T04:31:35.063285+0000","last_clean":"2023-10-06T04:31:35.063285+0000","last_became_active":"2023-10-06T04:31:35.062975+0000","last_became_peered":"2023-10-06T04:31:35.062975+0000","last_unstale":"2023-10-06T04:31:35.063285+0000","last_undegraded":"2023-10-06T04:31:35.063285+0000","last_fullsized":"2023-10-06T04:31:35.063285+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058231+0000","last_change":"2023-10-06T04:31:35.058231+0000","last_active":"2023-10-06T04:31:35.058231+0000","last_peered":"2023-10-06T04:31:35.058231+0000","last_clean":"2023-10-06T04:31:35.058231+0000","last_became_active":"2023-10-06T04:31:35.057944+0000","last_became_peered":"2023-10-06T04:31:35.057944+0000","last_unstale":"2023-10-06T04:31:35.058231+0000","last_undegraded":"2023-10-06T04:31:35.058231+0000","last_fullsized":"2023-10-06T04:31:35.058231+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063120+0000","last_change":"2023-10-06T04:31:35.063120+0000","last_active":"2023-10-06T04:31:35.063120+0000","last_peered":"2023-10-06T04:31:35.063120+0000","last_clean":"2023-10-06T04:31:35.063120+0000","last_became_active":"2023-10-06T04:31:35.062868+0000","last_became_peered":"2023-10-06T04:31:35.062868+0000","last_unstale":"2023-10-06T04:31:35.063120+0000","last_undegraded":"2023-10-06T04:31:35.063120+0000","last_fullsized":"2023-10-06T04:31:35.063120+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046895+0000","last_change":"2023-10-06T04:31:34.042620+0000","last_active":"2023-10-06T04:31:35.046895+0000","last_peered":"2023-10-06T04:31:35.046895+0000","last_clean":"2023-10-06T04:31:35.046895+0000","last_became_active":"2023-10-06T04:31:34.042023+0000","last_became_peered":"2023-10-06T04:31:34.042023+0000","last_unstale":"2023-10-06T04:31:35.046895+0000","last_undegraded":"2023-10-06T04:31:35.046895+0000","last_fullsized":"2023-10-06T04:31:35.046895+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060955+0000","last_change":"2023-10-06T04:31:35.060955+0000","last_active":"2023-10-06T04:31:35.060955+0000","last_peered":"2023-10-06T04:31:35.060955+0000","last_clean":"2023-10-06T04:31:35.060955+0000","last_became_active":"2023-10-06T04:31:35.060124+0000","last_became_peered":"2023-10-06T04:31:35.060124+0000","last_unstale":"2023-10-06T04:31:35.060955+0000","last_undegraded":"2023-10-06T04:31:35.060955+0000","last_fullsized":"2023-10-06T04:31:35.060955+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.040490+0000","last_change":"2023-10-06T04:31:34.036811+0000","last_active":"2023-10-06T04:31:34.040490+0000","last_peered":"2023-10-06T04:31:34.040490+0000","last_clean":"2023-10-06T04:31:34.040490+0000","last_became_active":"2023-10-06T04:31:34.036513+0000","last_became_peered":"2023-10-06T04:31:34.036513+0000","last_unstale":"2023-10-06T04:31:34.040490+0000","last_undegraded":"2023-10-06T04:31:34.040490+0000","last_fullsized":"2023-10-06T04:31:34.040490+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055379+0000","last_change":"2023-10-06T04:31:35.055379+0000","last_active":"2023-10-06T04:31:35.055379+0000","last_peered":"2023-10-06T04:31:35.055379+0000","last_clean":"2023-10-06T04:31:35.055379+0000","last_became_active":"2023-10-06T04:31:35.055088+0000","last_became_peered":"2023-10-06T04:31:35.055088+0000","last_unstale":"2023-10-06T04:31:35.055379+0000","last_undegraded":"2023-10-06T04:31:35.055379+0000","last_fullsized":"2023-10-06T04:31:35.055379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.057492+0000","last_change":"2023-10-06T04:31:34.054496+0000","last_active":"2023-10-06T04:31:34.057492+0000","last_peered":"2023-10-06T04:31:34.057492+0000","last_clean":"2023-10-06T04:31:34.057492+0000","last_became_active":"2023-10-06T04:31:34.054225+0000","last_became_peered":"2023-10-06T04:31:34.054225+0000","last_unstale":"2023-10-06T04:31:34.057492+0000","last_undegraded":"2023-10-06T04:31:34.057492+0000","last_fullsized":"2023-10-06T04:31:34.057492+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058805+0000","last_change":"2023-10-06T04:31:35.058805+0000","last_active":"2023-10-06T04:31:35.058805+0000","last_peered":"2023-10-06T04:31:35.058805+0000","last_clean":"2023-10-06T04:31:35.058805+0000","last_became_active":"2023-10-06T04:31:35.058524+0000","last_became_peered":"2023-10-06T04:31:35.058524+0000","last_unstale":"2023-10-06T04:31:35.058805+0000","last_undegraded":"2023-10-06T04:31:35.058805+0000","last_fullsized":"2023-10-06T04:31:35.058805+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051512+0000","last_change":"2023-10-06T04:31:34.042810+0000","last_active":"2023-10-06T04:31:34.051512+0000","last_peered":"2023-10-06T04:31:34.051512+0000","last_clean":"2023-10-06T04:31:34.051512+0000","last_became_active":"2023-10-06T04:31:34.040519+0000","last_became_peered":"2023-10-06T04:31:34.040519+0000","last_unstale":"2023-10-06T04:31:34.051512+0000","last_undegraded":"2023-10-06T04:31:34.051512+0000","last_fullsized":"2023-10-06T04:31:34.051512+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061039+0000","last_change":"2023-10-06T04:31:35.061039+0000","last_active":"2023-10-06T04:31:35.061039+0000","last_peered":"2023-10-06T04:31:35.061039+0000","last_clean":"2023-10-06T04:31:35.061039+0000","last_became_active":"2023-10-06T04:31:35.060281+0000","last_became_peered":"2023-10-06T04:31:35.060281+0000","last_unstale":"2023-10-06T04:31:35.061039+0000","last_undegraded":"2023-10-06T04:31:35.061039+0000","last_fullsized":"2023-10-06T04:31:35.061039+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.047409+0000","last_change":"2023-10-06T04:31:34.039478+0000","last_active":"2023-10-06T04:31:34.047409+0000","last_peered":"2023-10-06T04:31:34.047409+0000","last_clean":"2023-10-06T04:31:34.047409+0000","last_became_active":"2023-10-06T04:31:34.039209+0000","last_became_peered":"2023-10-06T04:31:34.039209+0000","last_unstale":"2023-10-06T04:31:34.047409+0000","last_undegraded":"2023-10-06T04:31:34.047409+0000","last_fullsized":"2023-10-06T04:31:34.047409+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063717+0000","last_change":"2023-10-06T04:31:35.063717+0000","last_active":"2023-10-06T04:31:35.063717+0000","last_peered":"2023-10-06T04:31:35.063717+0000","last_clean":"2023-10-06T04:31:35.063717+0000","last_became_active":"2023-10-06T04:31:35.060241+0000","last_became_peered":"2023-10-06T04:31:35.060241+0000","last_unstale":"2023-10-06T04:31:35.063717+0000","last_undegraded":"2023-10-06T04:31:35.063717+0000","last_fullsized":"2023-10-06T04:31:35.063717+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.252505+0000","last_change":"2023-10-06T04:31:34.045777+0000","last_active":"2023-10-06T04:31:35.252505+0000","last_peered":"2023-10-06T04:31:35.252505+0000","last_clean":"2023-10-06T04:31:35.252505+0000","last_became_active":"2023-10-06T04:31:34.045498+0000","last_became_peered":"2023-10-06T04:31:34.045498+0000","last_unstale":"2023-10-06T04:31:35.252505+0000","last_undegraded":"2023-10-06T04:31:35.252505+0000","last_fullsized":"2023-10-06T04:31:35.252505+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.139941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063439+0000","last_change":"2023-10-06T04:31:35.063439+0000","last_active":"2023-10-06T04:31:35.063439+0000","last_peered":"2023-10-06T04:31:35.063439+0000","last_clean":"2023-10-06T04:31:35.063439+0000","last_became_active":"2023-10-06T04:31:35.063117+0000","last_became_peered":"2023-10-06T04:31:35.063117+0000","last_unstale":"2023-10-06T04:31:35.063439+0000","last_undegraded":"2023-10-06T04:31:35.063439+0000","last_fullsized":"2023-10-06T04:31:35.063439+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:16.144634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":15,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257849+0000","last_change":"2023-10-06T04:31:34.039091+0000","last_active":"2023-10-06T04:31:35.257849+0000","last_peered":"2023-10-06T04:31:35.257849+0000","last_clean":"2023-10-06T04:31:35.257849+0000","last_became_active":"2023-10-06T04:31:34.038778+0000","last_became_peered":"2023-10-06T04:31:34.038778+0000","last_unstale":"2023-10-06T04:31:35.257849+0000","last_undegraded":"2023-10-06T04:31:35.257849+0000","last_fullsized":"2023-10-06T04:31:35.257849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052851+0000","last_change":"2023-10-06T04:31:35.052851+0000","last_active":"2023-10-06T04:31:35.052851+0000","last_peered":"2023-10-06T04:31:35.052851+0000","last_clean":"2023-10-06T04:31:35.052851+0000","last_became_active":"2023-10-06T04:31:35.052560+0000","last_became_peered":"2023-10-06T04:31:35.052560+0000","last_unstale":"2023-10-06T04:31:35.052851+0000","last_undegraded":"2023-10-06T04:31:35.052851+0000","last_fullsized":"2023-10-06T04:31:35.052851+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:39.105270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.254917+0000","last_change":"2023-10-06T04:31:34.046053+0000","last_active":"2023-10-06T04:31:35.254917+0000","last_peered":"2023-10-06T04:31:35.254917+0000","last_clean":"2023-10-06T04:31:35.254917+0000","last_became_active":"2023-10-06T04:31:34.045776+0000","last_became_peered":"2023-10-06T04:31:34.045776+0000","last_unstale":"2023-10-06T04:31:35.254917+0000","last_undegraded":"2023-10-06T04:31:35.254917+0000","last_fullsized":"2023-10-06T04:31:35.254917+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:40.537300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062504+0000","last_change":"2023-10-06T04:31:35.062504+0000","last_active":"2023-10-06T04:31:35.062504+0000","last_peered":"2023-10-06T04:31:35.062504+0000","last_clean":"2023-10-06T04:31:35.062504+0000","last_became_active":"2023-10-06T04:31:35.060884+0000","last_became_peered":"2023-10-06T04:31:35.060884+0000","last_unstale":"2023-10-06T04:31:35.062504+0000","last_undegraded":"2023-10-06T04:31:35.062504+0000","last_fullsized":"2023-10-06T04:31:35.062504+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051578+0000","last_change":"2023-10-06T04:31:34.042234+0000","last_active":"2023-10-06T04:31:34.051578+0000","last_peered":"2023-10-06T04:31:34.051578+0000","last_clean":"2023-10-06T04:31:34.051578+0000","last_became_active":"2023-10-06T04:31:34.041959+0000","last_became_peered":"2023-10-06T04:31:34.041959+0000","last_unstale":"2023-10-06T04:31:34.051578+0000","last_undegraded":"2023-10-06T04:31:34.051578+0000","last_fullsized":"2023-10-06T04:31:34.051578+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061678+0000","last_change":"2023-10-06T04:31:35.061678+0000","last_active":"2023-10-06T04:31:35.061678+0000","last_peered":"2023-10-06T04:31:35.061678+0000","last_clean":"2023-10-06T04:31:35.061678+0000","last_became_active":"2023-10-06T04:31:35.061383+0000","last_became_peered":"2023-10-06T04:31:35.061383+0000","last_unstale":"2023-10-06T04:31:35.061678+0000","last_undegraded":"2023-10-06T04:31:35.061678+0000","last_fullsized":"2023-10-06T04:31:35.061678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.048210+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:34.048210+0000","last_peered":"2023-10-06T04:31:34.048210+0000","last_clean":"2023-10-06T04:31:34.048210+0000","last_became_active":"2023-10-06T04:31:34.046107+0000","last_became_peered":"2023-10-06T04:31:34.046107+0000","last_unstale":"2023-10-06T04:31:34.048210+0000","last_undegraded":"2023-10-06T04:31:34.048210+0000","last_fullsized":"2023-10-06T04:31:34.048210+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055168+0000","last_change":"2023-10-06T04:31:35.055168+0000","last_active":"2023-10-06T04:31:35.055168+0000","last_peered":"2023-10-06T04:31:35.055168+0000","last_clean":"2023-10-06T04:31:35.055168+0000","last_became_active":"2023-10-06T04:31:35.054903+0000","last_became_peered":"2023-10-06T04:31:35.054903+0000","last_unstale":"2023-10-06T04:31:35.055168+0000","last_undegraded":"2023-10-06T04:31:35.055168+0000","last_fullsized":"2023-10-06T04:31:35.055168+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":17,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257431+0000","last_change":"2023-10-06T04:31:34.039233+0000","last_active":"2023-10-06T04:31:35.257431+0000","last_peered":"2023-10-06T04:31:35.257431+0000","last_clean":"2023-10-06T04:31:35.257431+0000","last_became_active":"2023-10-06T04:31:34.038916+0000","last_became_peered":"2023-10-06T04:31:34.038916+0000","last_unstale":"2023-10-06T04:31:35.257431+0000","last_undegraded":"2023-10-06T04:31:35.257431+0000","last_fullsized":"2023-10-06T04:31:35.257431+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:31.215430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062145+0000","last_change":"2023-10-06T04:31:35.062145+0000","last_active":"2023-10-06T04:31:35.062145+0000","last_peered":"2023-10-06T04:31:35.062145+0000","last_clean":"2023-10-06T04:31:35.062145+0000","last_became_active":"2023-10-06T04:31:35.061180+0000","last_became_peered":"2023-10-06T04:31:35.061180+0000","last_unstale":"2023-10-06T04:31:35.062145+0000","last_undegraded":"2023-10-06T04:31:35.062145+0000","last_fullsized":"2023-10-06T04:31:35.062145+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050141+0000","last_change":"2023-10-06T04:31:34.043099+0000","last_active":"2023-10-06T04:31:35.050141+0000","last_peered":"2023-10-06T04:31:35.050141+0000","last_clean":"2023-10-06T04:31:35.050141+0000","last_became_active":"2023-10-06T04:31:34.042869+0000","last_became_peered":"2023-10-06T04:31:34.042869+0000","last_unstale":"2023-10-06T04:31:35.050141+0000","last_undegraded":"2023-10-06T04:31:35.050141+0000","last_fullsized":"2023-10-06T04:31:35.050141+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061606+0000","last_change":"2023-10-06T04:31:35.061606+0000","last_active":"2023-10-06T04:31:35.061606+0000","last_peered":"2023-10-06T04:31:35.061606+0000","last_clean":"2023-10-06T04:31:35.061606+0000","last_became_active":"2023-10-06T04:31:35.061333+0000","last_became_peered":"2023-10-06T04:31:35.061333+0000","last_unstale":"2023-10-06T04:31:35.061606+0000","last_undegraded":"2023-10-06T04:31:35.061606+0000","last_fullsized":"2023-10-06T04:31:35.061606+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255026+0000","last_change":"2023-10-06T04:31:34.045981+0000","last_active":"2023-10-06T04:31:35.255026+0000","last_peered":"2023-10-06T04:31:35.255026+0000","last_clean":"2023-10-06T04:31:35.255026+0000","last_became_active":"2023-10-06T04:31:34.045683+0000","last_became_peered":"2023-10-06T04:31:34.045683+0000","last_unstale":"2023-10-06T04:31:35.255026+0000","last_undegraded":"2023-10-06T04:31:35.255026+0000","last_fullsized":"2023-10-06T04:31:35.255026+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:22.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059429+0000","last_change":"2023-10-06T04:31:35.059429+0000","last_active":"2023-10-06T04:31:35.059429+0000","last_peered":"2023-10-06T04:31:35.059429+0000","last_clean":"2023-10-06T04:31:35.059429+0000","last_became_active":"2023-10-06T04:31:35.058829+0000","last_became_peered":"2023-10-06T04:31:35.058829+0000","last_unstale":"2023-10-06T04:31:35.059429+0000","last_undegraded":"2023-10-06T04:31:35.059429+0000","last_fullsized":"2023-10-06T04:31:35.059429+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049513+0000","last_change":"2023-10-06T04:31:34.048038+0000","last_active":"2023-10-06T04:31:35.049513+0000","last_peered":"2023-10-06T04:31:35.049513+0000","last_clean":"2023-10-06T04:31:35.049513+0000","last_became_active":"2023-10-06T04:31:34.047780+0000","last_became_peered":"2023-10-06T04:31:34.047780+0000","last_unstale":"2023-10-06T04:31:35.049513+0000","last_undegraded":"2023-10-06T04:31:35.049513+0000","last_fullsized":"2023-10-06T04:31:35.049513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:21.519366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057281+0000","last_change":"2023-10-06T04:31:35.057281+0000","last_active":"2023-10-06T04:31:35.057281+0000","last_peered":"2023-10-06T04:31:35.057281+0000","last_clean":"2023-10-06T04:31:35.057281+0000","last_became_active":"2023-10-06T04:31:35.056855+0000","last_became_peered":"2023-10-06T04:31:35.056855+0000","last_unstale":"2023-10-06T04:31:35.057281+0000","last_undegraded":"2023-10-06T04:31:35.057281+0000","last_fullsized":"2023-10-06T04:31:35.057281+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:23.765556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.048027+0000","last_change":"2023-10-06T04:31:34.046008+0000","last_active":"2023-10-06T04:31:34.048027+0000","last_peered":"2023-10-06T04:31:34.048027+0000","last_clean":"2023-10-06T04:31:34.048027+0000","last_became_active":"2023-10-06T04:31:34.045732+0000","last_became_peered":"2023-10-06T04:31:34.045732+0000","last_unstale":"2023-10-06T04:31:34.048027+0000","last_undegraded":"2023-10-06T04:31:34.048027+0000","last_fullsized":"2023-10-06T04:31:34.048027+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059341+0000","last_change":"2023-10-06T04:31:35.059341+0000","last_active":"2023-10-06T04:31:35.059341+0000","last_peered":"2023-10-06T04:31:35.059341+0000","last_clean":"2023-10-06T04:31:35.059341+0000","last_became_active":"2023-10-06T04:31:35.059087+0000","last_became_peered":"2023-10-06T04:31:35.059087+0000","last_unstale":"2023-10-06T04:31:35.059341+0000","last_undegraded":"2023-10-06T04:31:35.059341+0000","last_fullsized":"2023-10-06T04:31:35.059341+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.249360+0000","last_change":"2023-10-06T04:31:34.045304+0000","last_active":"2023-10-06T04:31:35.249360+0000","last_peered":"2023-10-06T04:31:35.249360+0000","last_clean":"2023-10-06T04:31:35.249360+0000","last_became_active":"2023-10-06T04:31:34.044891+0000","last_became_peered":"2023-10-06T04:31:34.044891+0000","last_unstale":"2023-10-06T04:31:35.249360+0000","last_undegraded":"2023-10-06T04:31:35.249360+0000","last_fullsized":"2023-10-06T04:31:35.249360+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055035+0000","last_change":"2023-10-06T04:31:35.055035+0000","last_active":"2023-10-06T04:31:35.055035+0000","last_peered":"2023-10-06T04:31:35.055035+0000","last_clean":"2023-10-06T04:31:35.055035+0000","last_became_active":"2023-10-06T04:31:35.054727+0000","last_became_peered":"2023-10-06T04:31:35.054727+0000","last_unstale":"2023-10-06T04:31:35.055035+0000","last_undegraded":"2023-10-06T04:31:35.055035+0000","last_fullsized":"2023-10-06T04:31:35.055035+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:31.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.046849+0000","last_change":"2023-10-06T04:31:34.041723+0000","last_active":"2023-10-06T04:31:34.046849+0000","last_peered":"2023-10-06T04:31:34.046849+0000","last_clean":"2023-10-06T04:31:34.046849+0000","last_became_active":"2023-10-06T04:31:34.041386+0000","last_became_peered":"2023-10-06T04:31:34.041386+0000","last_unstale":"2023-10-06T04:31:34.046849+0000","last_undegraded":"2023-10-06T04:31:34.046849+0000","last_fullsized":"2023-10-06T04:31:34.046849+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:58.993061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057831+0000","last_change":"2023-10-06T04:31:35.057831+0000","last_active":"2023-10-06T04:31:35.057831+0000","last_peered":"2023-10-06T04:31:35.057831+0000","last_clean":"2023-10-06T04:31:35.057831+0000","last_became_active":"2023-10-06T04:31:35.057257+0000","last_became_peered":"2023-10-06T04:31:35.057257+0000","last_unstale":"2023-10-06T04:31:35.057831+0000","last_undegraded":"2023-10-06T04:31:35.057831+0000","last_fullsized":"2023-10-06T04:31:35.057831+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.040032+0000","last_change":"2023-10-06T04:31:34.035016+0000","last_active":"2023-10-06T04:31:34.040032+0000","last_peered":"2023-10-06T04:31:34.040032+0000","last_clean":"2023-10-06T04:31:34.040032+0000","last_became_active":"2023-10-06T04:31:34.034721+0000","last_became_peered":"2023-10-06T04:31:34.034721+0000","last_unstale":"2023-10-06T04:31:34.040032+0000","last_undegraded":"2023-10-06T04:31:34.040032+0000","last_fullsized":"2023-10-06T04:31:34.040032+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049984+0000","last_change":"2023-10-06T04:31:35.049984+0000","last_active":"2023-10-06T04:31:35.049984+0000","last_peered":"2023-10-06T04:31:35.049984+0000","last_clean":"2023-10-06T04:31:35.049984+0000","last_became_active":"2023-10-06T04:31:35.049687+0000","last_became_peered":"2023-10-06T04:31:35.049687+0000","last_unstale":"2023-10-06T04:31:35.049984+0000","last_undegraded":"2023-10-06T04:31:35.049984+0000","last_fullsized":"2023-10-06T04:31:35.049984+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051973+0000","last_change":"2023-10-06T04:31:34.046626+0000","last_active":"2023-10-06T04:31:34.051973+0000","last_peered":"2023-10-06T04:31:34.051973+0000","last_clean":"2023-10-06T04:31:34.051973+0000","last_became_active":"2023-10-06T04:31:34.044656+0000","last_became_peered":"2023-10-06T04:31:34.044656+0000","last_unstale":"2023-10-06T04:31:34.051973+0000","last_undegraded":"2023-10-06T04:31:34.051973+0000","last_fullsized":"2023-10-06T04:31:34.051973+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059496+0000","last_change":"2023-10-06T04:31:35.059496+0000","last_active":"2023-10-06T04:31:35.059496+0000","last_peered":"2023-10-06T04:31:35.059496+0000","last_clean":"2023-10-06T04:31:35.059496+0000","last_became_active":"2023-10-06T04:31:35.059187+0000","last_became_peered":"2023-10-06T04:31:35.059187+0000","last_unstale":"2023-10-06T04:31:35.059496+0000","last_undegraded":"2023-10-06T04:31:35.059496+0000","last_fullsized":"2023-10-06T04:31:35.059496+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.037449+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:31:34.037449+0000","last_peered":"2023-10-06T04:31:34.037449+0000","last_clean":"2023-10-06T04:31:34.037449+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:31:34.037449+0000","last_undegraded":"2023-10-06T04:31:34.037449+0000","last_fullsized":"2023-10-06T04:31:34.037449+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.040196+0000","last_change":"2023-10-06T04:31:34.036361+0000","last_active":"2023-10-06T04:31:34.040196+0000","last_peered":"2023-10-06T04:31:34.040196+0000","last_clean":"2023-10-06T04:31:34.040196+0000","last_became_active":"2023-10-06T04:31:34.036048+0000","last_became_peered":"2023-10-06T04:31:34.036048+0000","last_unstale":"2023-10-06T04:31:34.040196+0000","last_undegraded":"2023-10-06T04:31:34.040196+0000","last_fullsized":"2023-10-06T04:31:34.040196+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061294+0000","last_change":"2023-10-06T04:31:35.061294+0000","last_active":"2023-10-06T04:31:35.061294+0000","last_peered":"2023-10-06T04:31:35.061294+0000","last_clean":"2023-10-06T04:31:35.061294+0000","last_became_active":"2023-10-06T04:31:35.061034+0000","last_became_peered":"2023-10-06T04:31:35.061034+0000","last_unstale":"2023-10-06T04:31:35.061294+0000","last_undegraded":"2023-10-06T04:31:35.061294+0000","last_fullsized":"2023-10-06T04:31:35.061294+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:59.355740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.030228+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:31:34.030228+0000","last_peered":"2023-10-06T04:31:34.030228+0000","last_clean":"2023-10-06T04:31:34.030228+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:31:34.030228+0000","last_undegraded":"2023-10-06T04:31:34.030228+0000","last_fullsized":"2023-10-06T04:31:34.030228+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.047752+0000","last_change":"2023-10-06T04:31:34.043758+0000","last_active":"2023-10-06T04:31:34.047752+0000","last_peered":"2023-10-06T04:31:34.047752+0000","last_clean":"2023-10-06T04:31:34.047752+0000","last_became_active":"2023-10-06T04:31:34.043386+0000","last_became_peered":"2023-10-06T04:31:34.043386+0000","last_unstale":"2023-10-06T04:31:34.047752+0000","last_undegraded":"2023-10-06T04:31:34.047752+0000","last_fullsized":"2023-10-06T04:31:34.047752+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058248+0000","last_change":"2023-10-06T04:31:35.058248+0000","last_active":"2023-10-06T04:31:35.058248+0000","last_peered":"2023-10-06T04:31:35.058248+0000","last_clean":"2023-10-06T04:31:35.058248+0000","last_became_active":"2023-10-06T04:31:35.057955+0000","last_became_peered":"2023-10-06T04:31:35.057955+0000","last_unstale":"2023-10-06T04:31:35.058248+0000","last_undegraded":"2023-10-06T04:31:35.058248+0000","last_fullsized":"2023-10-06T04:31:35.058248+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.027787+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:31:34.027787+0000","last_peered":"2023-10-06T04:31:34.027787+0000","last_clean":"2023-10-06T04:31:34.027787+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:31:34.027787+0000","last_undegraded":"2023-10-06T04:31:34.027787+0000","last_fullsized":"2023-10-06T04:31:34.027787+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.044046+0000","last_change":"2023-10-06T04:31:34.041900+0000","last_active":"2023-10-06T04:31:34.044046+0000","last_peered":"2023-10-06T04:31:34.044046+0000","last_clean":"2023-10-06T04:31:34.044046+0000","last_became_active":"2023-10-06T04:31:34.041621+0000","last_became_peered":"2023-10-06T04:31:34.041621+0000","last_unstale":"2023-10-06T04:31:34.044046+0000","last_undegraded":"2023-10-06T04:31:34.044046+0000","last_fullsized":"2023-10-06T04:31:34.044046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060634+0000","last_change":"2023-10-06T04:31:35.060634+0000","last_active":"2023-10-06T04:31:35.060634+0000","last_peered":"2023-10-06T04:31:35.060634+0000","last_clean":"2023-10-06T04:31:35.060634+0000","last_became_active":"2023-10-06T04:31:35.060183+0000","last_became_peered":"2023-10-06T04:31:35.060183+0000","last_unstale":"2023-10-06T04:31:35.060634+0000","last_undegraded":"2023-10-06T04:31:35.060634+0000","last_fullsized":"2023-10-06T04:31:35.060634+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.030112+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:31:34.030112+0000","last_peered":"2023-10-06T04:31:34.030112+0000","last_clean":"2023-10-06T04:31:34.030112+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:31:34.030112+0000","last_undegraded":"2023-10-06T04:31:34.030112+0000","last_fullsized":"2023-10-06T04:31:34.030112+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":165,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.036078+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:31:34.036078+0000","last_peered":"2023-10-06T04:31:34.036078+0000","last_clean":"2023-10-06T04:31:34.036078+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:31:34.036078+0000","last_undegraded":"2023-10-06T04:31:34.036078+0000","last_fullsized":"2023-10-06T04:31:34.036078+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.044001+0000","last_change":"2023-10-06T04:31:34.039806+0000","last_active":"2023-10-06T04:31:34.044001+0000","last_peered":"2023-10-06T04:31:34.044001+0000","last_clean":"2023-10-06T04:31:34.044001+0000","last_became_active":"2023-10-06T04:31:34.039496+0000","last_became_peered":"2023-10-06T04:31:34.039496+0000","last_unstale":"2023-10-06T04:31:34.044001+0000","last_undegraded":"2023-10-06T04:31:34.044001+0000","last_fullsized":"2023-10-06T04:31:34.044001+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050727+0000","last_change":"2023-10-06T04:31:35.050727+0000","last_active":"2023-10-06T04:31:35.050727+0000","last_peered":"2023-10-06T04:31:35.050727+0000","last_clean":"2023-10-06T04:31:35.050727+0000","last_became_active":"2023-10-06T04:31:35.050446+0000","last_became_peered":"2023-10-06T04:31:35.050446+0000","last_unstale":"2023-10-06T04:31:35.050727+0000","last_undegraded":"2023-10-06T04:31:35.050727+0000","last_fullsized":"2023-10-06T04:31:35.050727+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:08.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.036161+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:31:34.036161+0000","last_peered":"2023-10-06T04:31:34.036161+0000","last_clean":"2023-10-06T04:31:34.036161+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:31:34.036161+0000","last_undegraded":"2023-10-06T04:31:34.036161+0000","last_fullsized":"2023-10-06T04:31:34.036161+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.042513+0000","last_change":"2023-10-06T04:31:34.039113+0000","last_active":"2023-10-06T04:31:34.042513+0000","last_peered":"2023-10-06T04:31:34.042513+0000","last_clean":"2023-10-06T04:31:34.042513+0000","last_became_active":"2023-10-06T04:31:34.038410+0000","last_became_peered":"2023-10-06T04:31:34.038410+0000","last_unstale":"2023-10-06T04:31:34.042513+0000","last_undegraded":"2023-10-06T04:31:34.042513+0000","last_fullsized":"2023-10-06T04:31:34.042513+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059901+0000","last_change":"2023-10-06T04:31:35.059901+0000","last_active":"2023-10-06T04:31:35.059901+0000","last_peered":"2023-10-06T04:31:35.059901+0000","last_clean":"2023-10-06T04:31:35.059901+0000","last_became_active":"2023-10-06T04:31:35.059461+0000","last_became_peered":"2023-10-06T04:31:35.059461+0000","last_unstale":"2023-10-06T04:31:35.059901+0000","last_undegraded":"2023-10-06T04:31:35.059901+0000","last_fullsized":"2023-10-06T04:31:35.059901+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.037722+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:31:34.037722+0000","last_peered":"2023-10-06T04:31:34.037722+0000","last_clean":"2023-10-06T04:31:34.037722+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:31:34.037722+0000","last_undegraded":"2023-10-06T04:31:34.037722+0000","last_fullsized":"2023-10-06T04:31:34.037722+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.044169+0000","last_change":"2023-10-06T04:31:34.042477+0000","last_active":"2023-10-06T04:31:34.044169+0000","last_peered":"2023-10-06T04:31:34.044169+0000","last_clean":"2023-10-06T04:31:34.044169+0000","last_became_active":"2023-10-06T04:31:34.041582+0000","last_became_peered":"2023-10-06T04:31:34.041582+0000","last_unstale":"2023-10-06T04:31:34.044169+0000","last_undegraded":"2023-10-06T04:31:34.044169+0000","last_fullsized":"2023-10-06T04:31:34.044169+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061904+0000","last_change":"2023-10-06T04:31:35.061904+0000","last_active":"2023-10-06T04:31:35.061904+0000","last_peered":"2023-10-06T04:31:35.061904+0000","last_clean":"2023-10-06T04:31:35.061904+0000","last_became_active":"2023-10-06T04:31:35.061655+0000","last_became_peered":"2023-10-06T04:31:35.061655+0000","last_unstale":"2023-10-06T04:31:35.061904+0000","last_undegraded":"2023-10-06T04:31:35.061904+0000","last_fullsized":"2023-10-06T04:31:35.061904+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255152+0000","last_change":"2023-10-06T04:31:34.042617+0000","last_active":"2023-10-06T04:31:35.255152+0000","last_peered":"2023-10-06T04:31:35.255152+0000","last_clean":"2023-10-06T04:31:35.255152+0000","last_became_active":"2023-10-06T04:31:34.042346+0000","last_became_peered":"2023-10-06T04:31:34.042346+0000","last_unstale":"2023-10-06T04:31:35.255152+0000","last_undegraded":"2023-10-06T04:31:35.255152+0000","last_fullsized":"2023-10-06T04:31:35.255152+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.037532+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:31:34.037532+0000","last_peered":"2023-10-06T04:31:34.037532+0000","last_clean":"2023-10-06T04:31:34.037532+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:31:34.037532+0000","last_undegraded":"2023-10-06T04:31:34.037532+0000","last_fullsized":"2023-10-06T04:31:34.037532+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060668+0000","last_change":"2023-10-06T04:31:35.060668+0000","last_active":"2023-10-06T04:31:35.060668+0000","last_peered":"2023-10-06T04:31:35.060668+0000","last_clean":"2023-10-06T04:31:35.060668+0000","last_became_active":"2023-10-06T04:31:35.059927+0000","last_became_peered":"2023-10-06T04:31:35.059927+0000","last_unstale":"2023-10-06T04:31:35.060668+0000","last_undegraded":"2023-10-06T04:31:35.060668+0000","last_fullsized":"2023-10-06T04:31:35.060668+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.030721+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:31:34.030721+0000","last_peered":"2023-10-06T04:31:34.030721+0000","last_clean":"2023-10-06T04:31:34.030721+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:31:34.030721+0000","last_undegraded":"2023-10-06T04:31:34.030721+0000","last_fullsized":"2023-10-06T04:31:34.030721+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-10-06T04:31:34.051614+0000","last_change":"2023-10-06T04:31:34.044177+0000","last_active":"2023-10-06T04:31:34.051614+0000","last_peered":"2023-10-06T04:31:34.051614+0000","last_clean":"2023-10-06T04:31:34.051614+0000","last_became_active":"2023-10-06T04:31:34.043921+0000","last_became_peered":"2023-10-06T04:31:34.043921+0000","last_unstale":"2023-10-06T04:31:34.051614+0000","last_undegraded":"2023-10-06T04:31:34.051614+0000","last_fullsized":"2023-10-06T04:31:34.051614+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059320+0000","last_change":"2023-10-06T04:31:35.059320+0000","last_active":"2023-10-06T04:31:35.059320+0000","last_peered":"2023-10-06T04:31:35.059320+0000","last_clean":"2023-10-06T04:31:35.059320+0000","last_became_active":"2023-10-06T04:31:35.058458+0000","last_became_peered":"2023-10-06T04:31:35.058458+0000","last_unstale":"2023-10-06T04:31:35.059320+0000","last_undegraded":"2023-10-06T04:31:35.059320+0000","last_fullsized":"2023-10-06T04:31:35.059320+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:06.614824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.045324+0000","last_change":"2023-10-06T04:31:34.043203+0000","last_active":"2023-10-06T04:31:35.045324+0000","last_peered":"2023-10-06T04:31:35.045324+0000","last_clean":"2023-10-06T04:31:35.045324+0000","last_became_active":"2023-10-06T04:31:34.042886+0000","last_became_peered":"2023-10-06T04:31:34.042886+0000","last_unstale":"2023-10-06T04:31:35.045324+0000","last_undegraded":"2023-10-06T04:31:35.045324+0000","last_fullsized":"2023-10-06T04:31:35.045324+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057021+0000","last_change":"2023-10-06T04:31:35.057021+0000","last_active":"2023-10-06T04:31:35.057021+0000","last_peered":"2023-10-06T04:31:35.057021+0000","last_clean":"2023-10-06T04:31:35.057021+0000","last_became_active":"2023-10-06T04:31:35.056770+0000","last_became_peered":"2023-10-06T04:31:35.056770+0000","last_unstale":"2023-10-06T04:31:35.057021+0000","last_undegraded":"2023-10-06T04:31:35.057021+0000","last_fullsized":"2023-10-06T04:31:35.057021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255029+0000","last_change":"2023-10-06T04:31:34.044080+0000","last_active":"2023-10-06T04:31:35.255029+0000","last_peered":"2023-10-06T04:31:35.255029+0000","last_clean":"2023-10-06T04:31:35.255029+0000","last_became_active":"2023-10-06T04:31:34.042225+0000","last_became_peered":"2023-10-06T04:31:34.042225+0000","last_unstale":"2023-10-06T04:31:35.255029+0000","last_undegraded":"2023-10-06T04:31:35.255029+0000","last_fullsized":"2023-10-06T04:31:35.255029+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:17.059659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055021+0000","last_change":"2023-10-06T04:31:35.055021+0000","last_active":"2023-10-06T04:31:35.055021+0000","last_peered":"2023-10-06T04:31:35.055021+0000","last_clean":"2023-10-06T04:31:35.055021+0000","last_became_active":"2023-10-06T04:31:35.054737+0000","last_became_peered":"2023-10-06T04:31:35.054737+0000","last_unstale":"2023-10-06T04:31:35.055021+0000","last_undegraded":"2023-10-06T04:31:35.055021+0000","last_fullsized":"2023-10-06T04:31:35.055021+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:26.087138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257207+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:35.257207+0000","last_peered":"2023-10-06T04:31:35.257207+0000","last_clean":"2023-10-06T04:31:35.257207+0000","last_became_active":"2023-10-06T04:31:34.041996+0000","last_became_peered":"2023-10-06T04:31:34.041996+0000","last_unstale":"2023-10-06T04:31:35.257207+0000","last_undegraded":"2023-10-06T04:31:35.257207+0000","last_fullsized":"2023-10-06T04:31:35.257207+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059718+0000","last_change":"2023-10-06T04:31:35.059718+0000","last_active":"2023-10-06T04:31:35.059718+0000","last_peered":"2023-10-06T04:31:35.059718+0000","last_clean":"2023-10-06T04:31:35.059718+0000","last_became_active":"2023-10-06T04:31:35.059438+0000","last_became_peered":"2023-10-06T04:31:35.059438+0000","last_unstale":"2023-10-06T04:31:35.059718+0000","last_undegraded":"2023-10-06T04:31:35.059718+0000","last_fullsized":"2023-10-06T04:31:35.059718+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.047916+0000","last_change":"2023-10-06T04:31:34.047689+0000","last_active":"2023-10-06T04:31:35.047916+0000","last_peered":"2023-10-06T04:31:35.047916+0000","last_clean":"2023-10-06T04:31:35.047916+0000","last_became_active":"2023-10-06T04:31:34.047311+0000","last_became_peered":"2023-10-06T04:31:34.047311+0000","last_unstale":"2023-10-06T04:31:35.047916+0000","last_undegraded":"2023-10-06T04:31:35.047916+0000","last_fullsized":"2023-10-06T04:31:35.047916+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055295+0000","last_change":"2023-10-06T04:31:35.055295+0000","last_active":"2023-10-06T04:31:35.055295+0000","last_peered":"2023-10-06T04:31:35.055295+0000","last_clean":"2023-10-06T04:31:35.055295+0000","last_became_active":"2023-10-06T04:31:35.055019+0000","last_became_peered":"2023-10-06T04:31:35.055019+0000","last_unstale":"2023-10-06T04:31:35.055295+0000","last_undegraded":"2023-10-06T04:31:35.055295+0000","last_fullsized":"2023-10-06T04:31:35.055295+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257565+0000","last_change":"2023-10-06T04:31:34.040784+0000","last_active":"2023-10-06T04:31:35.257565+0000","last_peered":"2023-10-06T04:31:35.257565+0000","last_clean":"2023-10-06T04:31:35.257565+0000","last_became_active":"2023-10-06T04:31:34.040127+0000","last_became_peered":"2023-10-06T04:31:34.040127+0000","last_unstale":"2023-10-06T04:31:35.257565+0000","last_undegraded":"2023-10-06T04:31:35.257565+0000","last_fullsized":"2023-10-06T04:31:35.257565+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052827+0000","last_change":"2023-10-06T04:31:35.052827+0000","last_active":"2023-10-06T04:31:35.052827+0000","last_peered":"2023-10-06T04:31:35.052827+0000","last_clean":"2023-10-06T04:31:35.052827+0000","last_became_active":"2023-10-06T04:31:35.052539+0000","last_became_peered":"2023-10-06T04:31:35.052539+0000","last_unstale":"2023-10-06T04:31:35.052827+0000","last_undegraded":"2023-10-06T04:31:35.052827+0000","last_fullsized":"2023-10-06T04:31:35.052827+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051004+0000","last_change":"2023-10-06T04:31:35.051004+0000","last_active":"2023-10-06T04:31:35.051004+0000","last_peered":"2023-10-06T04:31:35.051004+0000","last_clean":"2023-10-06T04:31:35.051004+0000","last_became_active":"2023-10-06T04:31:35.050694+0000","last_became_peered":"2023-10-06T04:31:35.050694+0000","last_unstale":"2023-10-06T04:31:35.051004+0000","last_undegraded":"2023-10-06T04:31:35.051004+0000","last_fullsized":"2023-10-06T04:31:35.051004+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062748+0000","last_change":"2023-10-06T04:31:35.062748+0000","last_active":"2023-10-06T04:31:35.062748+0000","last_peered":"2023-10-06T04:31:35.062748+0000","last_clean":"2023-10-06T04:31:35.062748+0000","last_became_active":"2023-10-06T04:31:35.060650+0000","last_became_peered":"2023-10-06T04:31:35.060650+0000","last_unstale":"2023-10-06T04:31:35.062748+0000","last_undegraded":"2023-10-06T04:31:35.062748+0000","last_fullsized":"2023-10-06T04:31:35.062748+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052675+0000","last_change":"2023-10-06T04:31:35.052675+0000","last_active":"2023-10-06T04:31:35.052675+0000","last_peered":"2023-10-06T04:31:35.052675+0000","last_clean":"2023-10-06T04:31:35.052675+0000","last_became_active":"2023-10-06T04:31:35.052379+0000","last_became_peered":"2023-10-06T04:31:35.052379+0000","last_unstale":"2023-10-06T04:31:35.052675+0000","last_undegraded":"2023-10-06T04:31:35.052675+0000","last_fullsized":"2023-10-06T04:31:35.052675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:40.270087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061097+0000","last_change":"2023-10-06T04:31:35.061097+0000","last_active":"2023-10-06T04:31:35.061097+0000","last_peered":"2023-10-06T04:31:35.061097+0000","last_clean":"2023-10-06T04:31:35.061097+0000","last_became_active":"2023-10-06T04:31:35.060817+0000","last_became_peered":"2023-10-06T04:31:35.060817+0000","last_unstale":"2023-10-06T04:31:35.061097+0000","last_undegraded":"2023-10-06T04:31:35.061097+0000","last_fullsized":"2023-10-06T04:31:35.061097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":8,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062003+0000","last_change":"2023-10-06T04:31:35.062003+0000","last_active":"2023-10-06T04:31:35.062003+0000","last_peered":"2023-10-06T04:31:35.062003+0000","last_clean":"2023-10-06T04:31:35.062003+0000","last_became_active":"2023-10-06T04:31:35.061553+0000","last_became_peered":"2023-10-06T04:31:35.061553+0000","last_unstale":"2023-10-06T04:31:35.062003+0000","last_undegraded":"2023-10-06T04:31:35.062003+0000","last_fullsized":"2023-10-06T04:31:35.062003+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298116,"kb_used_data":828,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073724,"statfs":{"total":96636764160,"available":96331493376,"internally_reserved":0,"allocated":847872,"data_stored":673906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56499999999999995}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71199999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69799999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81999999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574853,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.62}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52300000000000002}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32800000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55300000000000005}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66100000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834574854,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298092,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073748,"statfs":{"total":96636764160,"available":96331517952,"internally_reserved":0,"allocated":823296,"data_stored":673046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33200000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36599999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.376}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47899999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50700000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":10,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28100000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30499999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38700000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73599999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.59099999999999997}]}]},{"osd":3,"up_from":13,"seq":55834574854,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30599999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32100000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.40100000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61099999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57199999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52700000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574855,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47799999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33300000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35799999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.434}]}]},{"osd":5,"up_from":13,"seq":55834574854,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297512,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074328,"statfs":{"total":96636764160,"available":96332111872,"internally_reserved":0,"allocated":229376,"data_stored":82678,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64800000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70799999999999996}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72199999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69299999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.503}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:31:39.677 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:31:39.677 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-10-06T04:31:39.677 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-10-06T04:31:39.677 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-10-06T04:31:39.678 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-10-06T04:31:39.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 Processor -- start 2023-10-06T04:31:39.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 -- start start 2023-10-06T04:31:39.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64f41591b0 0x7f64f4159580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b05b0 0x7f64f40b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f64f40b0a20 0x7f64f40b41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f415b370 con 0x7f64f41591b0 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64f40b0fe0 con 0x7f64f40b0a20 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f97d8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f40b12c0 con 0x7f64f40b05b0 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.826+0000 7f64f2ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64f41591b0 0x7f64f4159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f64f41591b0 0x7f64f4159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52312/0 (socket says 172.21.15.123:52312) 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 -- 172.21.15.123:0/782621749 learned_addr learned my addr 172.21.15.123:0/782621749 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f37fe700 1 --2- 172.21.15.123:0/782621749 >> v2:172.21.15.177:3300/0 conn(0x7f64f40b0a20 0x7f64f40b41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f27fc700 1 --2- 172.21.15.123:0/782621749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b05b0 0x7f64f40b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 -- 172.21.15.123:0/782621749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b05b0 msgr2=0x7f64f40b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 --2- 172.21.15.123:0/782621749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b05b0 0x7f64f40b3ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 -- 172.21.15.123:0/782621749 >> v2:172.21.15.177:3300/0 conn(0x7f64f40b0a20 msgr2=0x7f64f40b41e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 --2- 172.21.15.123:0/782621749 >> v2:172.21.15.177:3300/0 conn(0x7f64f40b0a20 0x7f64f40b41e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 -- 172.21.15.123:0/782621749 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f40b15a0 con 0x7f64f41591b0 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f2ffd700 1 --2- 172.21.15.123:0/782621749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f41591b0 0x7f64f4159580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64e40099b0 tx=0x7f64e4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60457a40e1a60323 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f1ffb700 1 -- 172.21.15.123:0/782621749 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400b820 con 0x7f64f41591b0 2023-10-06T04:31:39.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f1ffb700 1 -- 172.21.15.123:0/782621749 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e400b980 con 0x7f64f41591b0 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.827+0000 7f64f1ffb700 1 -- 172.21.15.123:0/782621749 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e400bd20 con 0x7f64f41591b0 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 -- 172.21.15.123:0/782621749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f41591b0 msgr2=0x7f64f4159580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 --2- 172.21.15.123:0/782621749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f41591b0 0x7f64f4159580 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f64e40099b0 tx=0x7f64e4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 -- 172.21.15.123:0/782621749 shutdown_connections 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 --2- 172.21.15.123:0/782621749 >> v2:172.21.15.177:3300/0 conn(0x7f64f40b0a20 0x7f64f40b41e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 --2- 172.21.15.123:0/782621749 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f41591b0 0x7f64f4159580 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 --2- 172.21.15.123:0/782621749 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b05b0 0x7f64f40b3ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 -- 172.21.15.123:0/782621749 >> 172.21.15.123:0/782621749 conn(0x7f64f40995a0 msgr2=0x7f64f4063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:39.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 -- 172.21.15.123:0/782621749 shutdown_connections 2023-10-06T04:31:39.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.828+0000 7f64f97d8700 1 -- 172.21.15.123:0/782621749 wait complete. 2023-10-06T04:31:39.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.829+0000 7f64f97d8700 1 Processor -- start 2023-10-06T04:31:39.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.829+0000 7f64f97d8700 1 -- start start 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f97d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f40b05b0 0x7f64f414aa30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f97d8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b0a20 0x7f64f40ad2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f97d8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f41591b0 0x7f64f4157430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f97d8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f416d7c0 con 0x7f64f40b05b0 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f97d8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f416ced0 con 0x7f64f41591b0 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f97d8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f416d170 con 0x7f64f40b0a20 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f40b05b0 0x7f64f414aa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f2ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f40b05b0 0x7f64f414aa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52328/0 (socket says 172.21.15.123:52328) 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.830+0000 7f64f2ffd700 1 -- 172.21.15.123:0/188711502 learned_addr learned my addr 172.21.15.123:0/188711502 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:39.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f27fc700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b0a20 0x7f64f40ad2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f37fe700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f41591b0 0x7f64f4157430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f2ffd700 1 -- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b0a20 msgr2=0x7f64f40ad2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f2ffd700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b0a20 0x7f64f40ad2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f2ffd700 1 -- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f41591b0 msgr2=0x7f64f4157430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f2ffd700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f41591b0 0x7f64f4157430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f2ffd700 1 -- 172.21.15.123:0/188711502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64e40094f0 con 0x7f64f40b05b0 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f27fc700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b0a20 0x7f64f40ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:39.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f2ffd700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f40b05b0 0x7f64f414aa30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f64e40094c0 tx=0x7f64e4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64daffd700 1 -- 172.21.15.123:0/188711502 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e4024440 con 0x7f64f40b05b0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64daffd700 1 -- 172.21.15.123:0/188711502 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64e40245e0 con 0x7f64f40b05b0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64f416d450 con 0x7f64f40b05b0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64daffd700 1 -- 172.21.15.123:0/188711502 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64e40249a0 con 0x7f64f40b05b0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.831+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64f4156c70 con 0x7f64f40b05b0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.832+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64f416daa0 con 0x7f64f40b05b0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.832+0000 7f64daffd700 1 -- 172.21.15.123:0/188711502 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64e4024d50 con 0x7f64f40b05b0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.833+0000 7f64daffd700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64e0066f30 0x7f64e00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.833+0000 7f64f27fc700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64e0066f30 0x7f64e00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:39.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.833+0000 7f64daffd700 1 -- 172.21.15.123:0/188711502 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f64e4086230 con 0x7f64f40b05b0 2023-10-06T04:31:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.833+0000 7f64f27fc700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64e0066f30 0x7f64e00693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64f40d1310 tx=0x7f64e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:39.840 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:39.837+0000 7f64daffd700 1 -- 172.21.15.123:0/188711502 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64e4053340 con 0x7f64f40b05b0 2023-10-06T04:31:40.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.070+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f64f40b1880 con 0x7f64f40b05b0 2023-10-06T04:31:40.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.072+0000 7f64daffd700 1 -- 172.21.15.123:0/188711502 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163709 (secure 0 0 0) 0x7f64e4056960 con 0x7f64f40b05b0 2023-10-06T04:31:40.075 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:40.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64e0066f30 msgr2=0x7f64e00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64e0066f30 0x7f64e00693e0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f64f40d1310 tx=0x7f64e8008040 comp rx=0 tx=0).stop 2023-10-06T04:31:40.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f40b05b0 msgr2=0x7f64f414aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f40b05b0 0x7f64f414aa30 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f64e40094c0 tx=0x7f64e4017000 comp rx=0 tx=0).stop 2023-10-06T04:31:40.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 shutdown_connections 2023-10-06T04:31:40.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64e0066f30 0x7f64e00693e0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64f40b05b0 0x7f64f414aa30 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64f41591b0 0x7f64f4157430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 --2- 172.21.15.123:0/188711502 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64f40b0a20 0x7f64f40ad2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 >> 172.21.15.123:0/188711502 conn(0x7f64f40995a0 msgr2=0x7f64f4068660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 shutdown_connections 2023-10-06T04:31:40.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.075+0000 7f64f97d8700 1 -- 172.21.15.123:0/188711502 wait complete. 2023-10-06T04:31:40.088 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6832","nonce":119418},{"type":"v1","addr":"172.21.15.123:6833","nonce":119418}]},"active_addr":"172.21.15.123:6833/119418","active_change":"2023-10-06T04:31:14.873582+0000","active_mgr_features":4540138320759226367,"available":true,"standbys":[{"gid":4099,"name":"x","mgr_features":4540138320759226367,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.1.5","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"drive_group_interval":{"name":"drive_group_interval","type":"float","level":"advanced","flags":0,"default_value":"300.0","min":"","max":"","enum_allowed":[],"desc":"interval in seconds between re-application of applied drive_groups","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":3278521103}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1421929417}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":4236349628}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.123:0","nonce":1333698400}]}]} 2023-10-06T04:31:40.092 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-10-06T04:31:40.093 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-10-06T04:31:40.093 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:40.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.210+0000 7f71c0a06700 1 Processor -- start 2023-10-06T04:31:40.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.210+0000 7f71c0a06700 1 -- start start 2023-10-06T04:31:40.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71c0a06700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f71bc06cd60 0x7f71bc0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71c0a06700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71bc157330 0x7f71bc0adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71c0a06700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71bc157910 con 0x7f71bc156f60 2023-10-06T04:31:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71c0a06700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc06d240 con 0x7f71bc157330 2023-10-06T04:31:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71c0a06700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71bc06d520 con 0x7f71bc06cd60 2023-10-06T04:31:40.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71bad9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71bc157330 0x7f71bc0adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71bad9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f71bc157330 0x7f71bc0adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52340/0 (socket says 172.21.15.123:52340) 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71bad9d700 1 -- 172.21.15.123:0/3766330018 learned_addr learned my addr 172.21.15.123:0/3766330018 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.211+0000 7f71b9d9b700 1 --2- 172.21.15.123:0/3766330018 >> v2:172.21.15.177:3300/0 conn(0x7f71bc06cd60 0x7f71bc0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71bad9d700 1 -- 172.21.15.123:0/3766330018 >> v2:172.21.15.177:3300/0 conn(0x7f71bc06cd60 msgr2=0x7f71bc0ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71bad9d700 1 --2- 172.21.15.123:0/3766330018 >> v2:172.21.15.177:3300/0 conn(0x7f71bc06cd60 0x7f71bc0ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71bad9d700 1 -- 172.21.15.123:0/3766330018 >> v1:172.21.15.177:6789/0 conn(0x7f71bc156f60 legacy=0x7f71bc15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71bad9d700 1 -- 172.21.15.123:0/3766330018 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71bc06d800 con 0x7f71bc157330 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71bad9d700 1 --2- 172.21.15.123:0/3766330018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0adc70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009f20 tx=0x7f71b4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3167d46d69b82621 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71b959a700 1 -- 172.21.15.123:0/3766330018 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b400e070 con 0x7f71bc157330 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71b959a700 1 -- 172.21.15.123:0/3766330018 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b400b990 con 0x7f71bc157330 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71b959a700 1 -- 172.21.15.123:0/3766330018 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40092e0 con 0x7f71bc157330 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71c0a06700 1 -- 172.21.15.123:0/3766330018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 msgr2=0x7f71bc0adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71c0a06700 1 --2- 172.21.15.123:0/3766330018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0adc70 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009f20 tx=0x7f71b4009840 comp rx=0 tx=0).stop 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71c0a06700 1 -- 172.21.15.123:0/3766330018 shutdown_connections 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71c0a06700 1 --2- 172.21.15.123:0/3766330018 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0adc70 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.212+0000 7f71c0a06700 1 --2- 172.21.15.123:0/3766330018 >> v2:172.21.15.177:3300/0 conn(0x7f71bc06cd60 0x7f71bc0ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.213+0000 7f71c0a06700 1 -- 172.21.15.123:0/3766330018 >> 172.21.15.123:0/3766330018 conn(0x7f71bc0ce0c0 msgr2=0x7f71bc0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.213+0000 7f71c0a06700 1 -- 172.21.15.123:0/3766330018 shutdown_connections 2023-10-06T04:31:40.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.213+0000 7f71c0a06700 1 -- 172.21.15.123:0/3766330018 wait complete. 2023-10-06T04:31:40.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.214+0000 7f71c0a06700 1 Processor -- start 2023-10-06T04:31:40.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.214+0000 7f71c0a06700 1 -- start start 2023-10-06T04:31:40.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71c0a06700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71bc06cd60 0x7f71bc0c0e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71c0a06700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc156f60 0x7f71bc0c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71c0a06700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71c0a06700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0c6660 con 0x7f71bc157330 2023-10-06T04:31:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71c0a06700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0c2b40 con 0x7f71bc156f60 2023-10-06T04:31:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71c0a06700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71bc0c1350 con 0x7f71bc06cd60 2023-10-06T04:31:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:52344/0 (socket says 172.21.15.123:52344) 2023-10-06T04:31:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 -- 172.21.15.123:0/927834815 learned_addr learned my addr 172.21.15.123:0/927834815 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71b9d9b700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc156f60 0x7f71bc0c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71ba59c700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71bc06cd60 0x7f71bc0c0e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 -- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71bc06cd60 msgr2=0x7f71bc0c0e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71bc06cd60 0x7f71bc0c0e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 -- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc156f60 msgr2=0x7f71bc0c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc156f60 0x7f71bc0c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 -- 172.21.15.123:0/927834815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71b4009ae0 con 0x7f71bc157330 2023-10-06T04:31:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71b9d9b700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc156f60 0x7f71bc0c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71ba59c700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71bc06cd60 0x7f71bc0c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.215+0000 7f71bad9d700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0af680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009880 tx=0x7f71b4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.216+0000 7f71aa7fc700 1 -- 172.21.15.123:0/927834815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b4008d00 con 0x7f71bc157330 2023-10-06T04:31:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.216+0000 7f71aa7fc700 1 -- 172.21.15.123:0/927834815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71b4008e60 con 0x7f71bc157330 2023-10-06T04:31:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.216+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71bc0c22a0 con 0x7f71bc157330 2023-10-06T04:31:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.216+0000 7f71aa7fc700 1 -- 172.21.15.123:0/927834815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71b40256b0 con 0x7f71bc157330 2023-10-06T04:31:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.216+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71bc0c2860 con 0x7f71bc157330 2023-10-06T04:31:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.217+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71bc06e690 con 0x7f71bc157330 2023-10-06T04:31:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.217+0000 7f71aa7fc700 1 -- 172.21.15.123:0/927834815 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f71b4003040 con 0x7f71bc157330 2023-10-06T04:31:40.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.217+0000 7f71aa7fc700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f719c066f30 0x7f719c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.217+0000 7f71aa7fc700 1 -- 172.21.15.123:0/927834815 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f71b4085d40 con 0x7f71bc157330 2023-10-06T04:31:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.217+0000 7f71ba59c700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f719c066f30 0x7f719c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.218+0000 7f71ba59c700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f719c066f30 0x7f719c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71ac007fd0 tx=0x7f71ac009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.222+0000 7f71aa7fc700 1 -- 172.21.15.123:0/927834815 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f71b4052e50 con 0x7f71bc157330 2023-10-06T04:31:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.403+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71bc06dae0 con 0x7f71bc157330 2023-10-06T04:31:40.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.404+0000 7f71aa7fc700 1 -- 172.21.15.123:0/927834815 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f71b4056470 con 0x7f71bc157330 2023-10-06T04:31:40.406 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:40.406 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:40.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f719c066f30 msgr2=0x7f719c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f719c066f30 0x7f719c0693e0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f71ac007fd0 tx=0x7f71ac009040 comp rx=0 tx=0).stop 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 msgr2=0x7f71bc0af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0af680 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f71b4009880 tx=0x7f71b4018000 comp rx=0 tx=0).stop 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 shutdown_connections 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f719c066f30 0x7f719c0693e0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f71bc157330 0x7f71bc0af680 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f71bc06cd60 0x7f71bc0c0e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 --2- 172.21.15.123:0/927834815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f71bc156f60 0x7f71bc0c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 >> 172.21.15.123:0/927834815 conn(0x7f71bc0ce0c0 msgr2=0x7f71bc0cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 shutdown_connections 2023-10-06T04:31:40.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.406+0000 7f71c0a06700 1 -- 172.21.15.123:0/927834815 wait complete. 2023-10-06T04:31:40.419 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-10-06T04:31:40.420 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:40.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.534+0000 7f7fe71e0700 1 Processor -- start 2023-10-06T04:31:40.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.534+0000 7f7fe71e0700 1 -- start start 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe71e0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7fe00ac720 0x7f7fe00ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe71e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00680b0 0x7f7fe00acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe71e0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7fe006b760 con 0x7f7fe006e250 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe71e0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7fe006deb0 con 0x7f7fe00ac720 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe71e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00aecc0 con 0x7f7fe00680b0 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe4f7c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7fe00ac720 0x7f7fe00ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe577d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7fe006e250 0x7f7fe006e6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56884/0 (socket says 172.21.15.123:56884) 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fe577d700 1 -- 172.21.15.123:0/4083690903 learned_addr learned my addr 172.21.15.123:0/4083690903 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:40.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.535+0000 7f7fdffff700 1 --2- 172.21.15.123:0/4083690903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00680b0 0x7f7fe00acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe4f7c700 1 -- 172.21.15.123:0/4083690903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00680b0 msgr2=0x7f7fe00acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe4f7c700 1 --2- 172.21.15.123:0/4083690903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00680b0 0x7f7fe00acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe4f7c700 1 -- 172.21.15.123:0/4083690903 >> v1:172.21.15.123:6789/0 conn(0x7f7fe006e250 legacy=0x7f7fe006e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe4f7c700 1 -- 172.21.15.123:0/4083690903 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fe00ad030 con 0x7f7fe00ac720 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe4f7c700 1 --2- 172.21.15.123:0/4083690903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00ac720 0x7f7fe00ae780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40098e0 tx=0x7f7fd40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c533bedc7d14e4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fdf7fe700 1 -- 172.21.15.123:0/4083690903 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd4009e80 con 0x7f7fe00ac720 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fdf7fe700 1 -- 172.21.15.123:0/4083690903 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd400b8f0 con 0x7f7fe00ac720 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fdf7fe700 1 -- 172.21.15.123:0/4083690903 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400bc70 con 0x7f7fe00ac720 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 -- 172.21.15.123:0/4083690903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00ac720 msgr2=0x7f7fe00ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/4083690903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00ac720 0x7f7fe00ae780 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f7fd40098e0 tx=0x7f7fd40091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 -- 172.21.15.123:0/4083690903 shutdown_connections 2023-10-06T04:31:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/4083690903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00ac720 0x7f7fe00ae780 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/4083690903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00680b0 0x7f7fe00acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 -- 172.21.15.123:0/4083690903 >> 172.21.15.123:0/4083690903 conn(0x7f7fe00cda80 msgr2=0x7f7fe00c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 -- 172.21.15.123:0/4083690903 shutdown_connections 2023-10-06T04:31:40.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.536+0000 7f7fe71e0700 1 -- 172.21.15.123:0/4083690903 wait complete. 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.537+0000 7f7fe71e0700 1 Processor -- start 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe71e0700 1 -- start start 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe71e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00680b0 0x7f7fe00c1670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe71e0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe006e250 0x7f7fe00c3690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe71e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00ac720 0x7f7fe01505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe71e0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00c1ec0 con 0x7f7fe00680b0 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe71e0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00c21a0 con 0x7f7fe006e250 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe71e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7fe00c2480 con 0x7f7fe00ac720 2023-10-06T04:31:40.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe4f7c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00680b0 0x7f7fe00c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe4f7c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00680b0 0x7f7fe00c1670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43136/0 (socket says 172.21.15.123:43136) 2023-10-06T04:31:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.538+0000 7f7fe4f7c700 1 -- 172.21.15.123:0/1038995322 learned_addr learned my addr 172.21.15.123:0/1038995322 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fdffff700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe006e250 0x7f7fe00c3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe577d700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00ac720 0x7f7fe01505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe4f7c700 1 -- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00ac720 msgr2=0x7f7fe01505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe4f7c700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00ac720 0x7f7fe01505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe4f7c700 1 -- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe006e250 msgr2=0x7f7fe00c3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe4f7c700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe006e250 0x7f7fe00c3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe4f7c700 1 -- 172.21.15.123:0/1038995322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7fd4009420 con 0x7f7fe00680b0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe4f7c700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00680b0 0x7f7fe00c1670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400b3c0 tx=0x7f7fd4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fdcff9700 1 -- 172.21.15.123:0/1038995322 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd400bc70 con 0x7f7fe00680b0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fdcff9700 1 -- 172.21.15.123:0/1038995322 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7fd4011420 con 0x7f7fe00680b0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7fe00c2760 con 0x7f7fe00680b0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fdcff9700 1 -- 172.21.15.123:0/1038995322 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7fd40117c0 con 0x7f7fe00680b0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.539+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7fe00c1bb0 con 0x7f7fe00680b0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.540+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7fe0152850 con 0x7f7fe00680b0 2023-10-06T04:31:40.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.540+0000 7f7fdcff9700 1 -- 172.21.15.123:0/1038995322 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7fd4011b70 con 0x7f7fe00680b0 2023-10-06T04:31:40.546 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.541+0000 7f7fdcff9700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7fc4066f30 0x7f7fc40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.546 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.541+0000 7f7fdcff9700 1 -- 172.21.15.123:0/1038995322 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7fd4086010 con 0x7f7fe00680b0 2023-10-06T04:31:40.546 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.541+0000 7f7fdffff700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7fc4066f30 0x7f7fc40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.546 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.541+0000 7f7fdffff700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7fc4066f30 0x7f7fc40693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc006fd0 tx=0x7f7fcc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.546 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.545+0000 7f7fdcff9700 1 -- 172.21.15.123:0/1038995322 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7fe0152850 con 0x7f7fe00680b0 2023-10-06T04:31:40.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.726+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7fe0152850 con 0x7f7fe00680b0 2023-10-06T04:31:40.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.727+0000 7f7fdcff9700 1 -- 172.21.15.123:0/1038995322 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f7fe0152850 con 0x7f7fe00680b0 2023-10-06T04:31:40.728 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:40.728 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.728+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7fc4066f30 msgr2=0x7f7fc40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.728+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7fc4066f30 0x7f7fc40693e0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc006fd0 tx=0x7f7fcc008040 comp rx=0 tx=0).stop 2023-10-06T04:31:40.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.728+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00680b0 msgr2=0x7f7fe00c1670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.728+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00680b0 0x7f7fe00c1670 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400b3c0 tx=0x7f7fd4017000 comp rx=0 tx=0).stop 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 shutdown_connections 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7fc4066f30 0x7f7fc40693e0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7fe00680b0 0x7f7fe00c1670 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7fe00ac720 0x7f7fe01505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 --2- 172.21.15.123:0/1038995322 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7fe006e250 0x7f7fe00c3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 >> 172.21.15.123:0/1038995322 conn(0x7f7fe00cda80 msgr2=0x7f7fe00afe30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 shutdown_connections 2023-10-06T04:31:40.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.729+0000 7f7fe71e0700 1 -- 172.21.15.123:0/1038995322 wait complete. 2023-10-06T04:31:40.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-10-06T04:31:40.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-10-06T04:31:40.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-10-06T04:31:40.743 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-10-06T04:31:40.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-10-06T04:31:40.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-10-06T04:31:40.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-10-06T04:31:40.745 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-10-06T04:31:40.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.944+0000 7f6ea3a19700 1 Processor -- start 2023-10-06T04:31:40.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.944+0000 7f6ea3a19700 1 -- start start 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.945+0000 7f6ea3a19700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6e9c15d5c0 0x7f6e9c0c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.945+0000 7f6ea3a19700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6e9c161aa0 0x7f6e9c0c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.945+0000 7f6ea3a19700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9c0c7ae0 con 0x7f6e9c06f180 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.945+0000 7f6ea3a19700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c0c6ed0 con 0x7f6e9c15d5c0 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea3a19700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9c0c71b0 con 0x7f6e9c161aa0 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea1fb6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6e9c161aa0 0x7f6e9c0c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea1fb6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6e9c161aa0 0x7f6e9c0c7f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:40942/0 (socket says 172.21.15.123:40942) 2023-10-06T04:31:40.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea1fb6700 1 -- 172.21.15.123:0/2418732654 learned_addr learned my addr 172.21.15.123:0/2418732654 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea1fb6700 1 -- 172.21.15.123:0/2418732654 >> v2:172.21.15.123:3300/0 conn(0x7f6e9c15d5c0 msgr2=0x7f6e9c0c66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea1fb6700 1 --2- 172.21.15.123:0/2418732654 >> v2:172.21.15.123:3300/0 conn(0x7f6e9c15d5c0 0x7f6e9c0c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea1fb6700 1 -- 172.21.15.123:0/2418732654 >> v1:172.21.15.123:6789/0 conn(0x7f6e9c06f180 legacy=0x7f6e9c161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.946+0000 7f6ea1fb6700 1 -- 172.21.15.123:0/2418732654 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9c0c7490 con 0x7f6e9c161aa0 2023-10-06T04:31:40.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.947+0000 7f6ea1fb6700 1 --2- 172.21.15.123:0/2418732654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c161aa0 0x7f6e9c0c7f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c06a7d0 tx=0x7f6e94009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9d7374f79870b02 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.947+0000 7f6e93fff700 1 -- 172.21.15.123:0/2418732654 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e9400e070 con 0x7f6e9c161aa0 2023-10-06T04:31:40.949 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.947+0000 7f6e93fff700 1 -- 172.21.15.123:0/2418732654 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e9400ba10 con 0x7f6e9c161aa0 2023-10-06T04:31:40.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.947+0000 7f6e93fff700 1 -- 172.21.15.123:0/2418732654 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e940092e0 con 0x7f6e9c161aa0 2023-10-06T04:31:40.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 -- 172.21.15.123:0/2418732654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c161aa0 msgr2=0x7f6e9c0c7f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 --2- 172.21.15.123:0/2418732654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c161aa0 0x7f6e9c0c7f70 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c06a7d0 tx=0x7f6e94009840 comp rx=0 tx=0).stop 2023-10-06T04:31:40.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 -- 172.21.15.123:0/2418732654 shutdown_connections 2023-10-06T04:31:40.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 --2- 172.21.15.123:0/2418732654 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c161aa0 0x7f6e9c0c7f70 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.950 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 --2- 172.21.15.123:0/2418732654 >> v2:172.21.15.123:3300/0 conn(0x7f6e9c15d5c0 0x7f6e9c0c66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.951 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 -- 172.21.15.123:0/2418732654 >> 172.21.15.123:0/2418732654 conn(0x7f6e9c067660 msgr2=0x7f6e9c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.951 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 -- 172.21.15.123:0/2418732654 shutdown_connections 2023-10-06T04:31:40.951 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.948+0000 7f6ea3a19700 1 -- 172.21.15.123:0/2418732654 wait complete. 2023-10-06T04:31:40.951 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.950+0000 7f6ea3a19700 1 Processor -- start 2023-10-06T04:31:40.951 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.950+0000 7f6ea3a19700 1 -- start start 2023-10-06T04:31:40.952 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.950+0000 7f807546f700 1 Processor -- start 2023-10-06T04:31:40.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f6ea3a19700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c06f180 0x7f6e9c0cc3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f6ea3a19700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e9c15d5c0 0x7f6e9c160db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f6ea3a19700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9c161aa0 0x7f6e9c0c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f6ea3a19700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15e890 con 0x7f6e9c161aa0 2023-10-06T04:31:40.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f6ea3a19700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c15eb70 con 0x7f6e9c06f180 2023-10-06T04:31:40.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f6ea3a19700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9c1602d0 con 0x7f6e9c15d5c0 2023-10-06T04:31:40.953 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c06f180 0x7f6e9c0cc3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f807546f700 1 -- start start 2023-10-06T04:31:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f807546f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80701592b0 0x7f80700ccd50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f807546f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f80700acea0 0x7f80700cd290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f807546f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f80700c5ba0 con 0x7f8070159680 2023-10-06T04:31:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f807546f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f80700c5270 con 0x7f80700acea0 2023-10-06T04:31:40.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.951+0000 7f807546f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80700c5550 con 0x7f80701592b0 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c06f180 0x7f6e9c0cc3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:40952/0 (socket says 172.21.15.123:40952) 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 -- 172.21.15.123:0/3856206091 learned_addr learned my addr 172.21.15.123:0/3856206091 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea0fb4700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e9c15d5c0 0x7f6e9c160db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 -- 172.21.15.123:0/3856206091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e9c15d5c0 msgr2=0x7f6e9c160db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6e9c15d5c0 0x7f6e9c160db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 -- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9c161aa0 msgr2=0x7f6e9c0c3dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6e9c161aa0 0x7f6e9c0c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 -- 172.21.15.123:0/3856206091 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e94009ae0 con 0x7f6e9c06f180 2023-10-06T04:31:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f806e7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f80700acea0 0x7f80700cd290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f806e7fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f80700acea0 0x7f80700cd290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:40964/0 (socket says 172.21.15.123:40964) 2023-10-06T04:31:40.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f806e7fc700 1 -- 172.21.15.123:0/946182487 learned_addr learned my addr 172.21.15.123:0/946182487 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea17b5700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c06f180 0x7f6e9c0cc3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00b810 tx=0x7f6e8c009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00f510 con 0x7f6e9c06f180 2023-10-06T04:31:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e8c00f690 con 0x7f6e9c06f180 2023-10-06T04:31:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e9c1605b0 con 0x7f6e9c06f180 2023-10-06T04:31:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.953+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e8c00fa50 con 0x7f6e9c06f180 2023-10-06T04:31:40.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.953+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e9c0cc930 con 0x7f6e9c06f180 2023-10-06T04:31:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f806e7fc700 1 -- 172.21.15.123:0/946182487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80701592b0 msgr2=0x7f80700ccd50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:31:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f806e7fc700 1 --2- 172.21.15.123:0/946182487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80701592b0 0x7f80700ccd50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.952+0000 7f806e7fc700 1 -- 172.21.15.123:0/946182487 >> v1:172.21.15.123:6789/0 conn(0x7f8070159680 legacy=0x7f80700c6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.953+0000 7f806e7fc700 1 -- 172.21.15.123:0/946182487 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80700c5830 con 0x7f80700acea0 2023-10-06T04:31:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.953+0000 7f806e7fc700 1 --2- 172.21.15.123:0/946182487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 0x7f80700cd290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80600099b0 tx=0x7f8060009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87e338d08dc581f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.953+0000 7f806dffb700 1 -- 172.21.15.123:0/946182487 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000b820 con 0x7f80700acea0 2023-10-06T04:31:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.953+0000 7fe5ddacc700 1 Processor -- start 2023-10-06T04:31:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6e9c160130 con 0x7f6e9c06f180 2023-10-06T04:31:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.953+0000 7f806dffb700 1 -- 172.21.15.123:0/946182487 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f806000b980 con 0x7f80700acea0 2023-10-06T04:31:40.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f806dffb700 1 -- 172.21.15.123:0/946182487 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f806000bd20 con 0x7f80700acea0 2023-10-06T04:31:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7fe5ddacc700 1 -- start start 2023-10-06T04:31:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 -- 172.21.15.123:0/946182487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 msgr2=0x7f80700cd290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 --2- 172.21.15.123:0/946182487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 0x7f80700cd290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f80600099b0 tx=0x7f8060009250 comp rx=0 tx=0).stop 2023-10-06T04:31:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 -- 172.21.15.123:0/946182487 shutdown_connections 2023-10-06T04:31:40.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 --2- 172.21.15.123:0/946182487 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f80701592b0 0x7f80700ccd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 --2- 172.21.15.123:0/946182487 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 0x7f80700cd290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 -- 172.21.15.123:0/946182487 >> 172.21.15.123:0/946182487 conn(0x7f80700995a0 msgr2=0x7f8070063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 -- 172.21.15.123:0/946182487 shutdown_connections 2023-10-06T04:31:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5ddacc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe5d80698c0 0x7fe5d80c6490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5ddacc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe5d80c43a0 0x7fe5d80c69d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5ddacc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d80c2f00 con 0x7fe5d80bf330 2023-10-06T04:31:40.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5ddacc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d80c7cf0 con 0x7fe5d80698c0 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5ddacc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d80c7fd0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5d7fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe5d80c43a0 0x7fe5d80c69d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5d77fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe5d80bf330 0x7fe5d80691b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56900/0 (socket says 172.21.15.123:56900) 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5d77fe700 1 -- 172.21.15.123:0/1274686471 learned_addr learned my addr 172.21.15.123:0/1274686471 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5d7fff700 1 -- 172.21.15.123:0/1274686471 >> v2:172.21.15.123:3300/0 conn(0x7fe5d80698c0 msgr2=0x7fe5d80c6490 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5d7fff700 1 --2- 172.21.15.123:0/1274686471 >> v2:172.21.15.123:3300/0 conn(0x7fe5d80698c0 0x7fe5d80c6490 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5d7fff700 1 -- 172.21.15.123:0/1274686471 >> v1:172.21.15.123:6789/0 conn(0x7fe5d80bf330 legacy=0x7fe5d80691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.955+0000 7fe5d7fff700 1 -- 172.21.15.123:0/1274686471 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d80c82b0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.956+0000 7fe5d7fff700 1 --2- 172.21.15.123:0/1274686471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80c43a0 0x7fe5d80c69d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80c8900 tx=0x7fe5d0009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=331cae38f3723f34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.956+0000 7fa89ea8e700 1 Processor -- start 2023-10-06T04:31:40.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.956+0000 7fa89ea8e700 1 -- start start 2023-10-06T04:31:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.954+0000 7f807546f700 1 -- 172.21.15.123:0/946182487 wait complete. 2023-10-06T04:31:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.956+0000 7f807546f700 1 Processor -- start 2023-10-06T04:31:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fe5d6ffd700 1 -- 172.21.15.123:0/1274686471 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000ba20 con 0x7fe5d80c43a0 2023-10-06T04:31:40.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fe5d6ffd700 1 -- 172.21.15.123:0/1274686471 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d001f070 con 0x7fe5d80c43a0 2023-10-06T04:31:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fe5d6ffd700 1 -- 172.21.15.123:0/1274686471 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d000bdd0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fa89ea8e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa8980adbc0 0x7fa8980c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fa89ea8e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8980ae050 con 0x7fa898156fe0 2023-10-06T04:31:40.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fa89ea8e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8980ae330 con 0x7fa8981573b0 2023-10-06T04:31:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fa89ea8e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8980692e0 con 0x7fa8980adbc0 2023-10-06T04:31:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fe5ddacc700 1 -- 172.21.15.123:0/1274686471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80c43a0 msgr2=0x7fe5d80c69d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fe5ddacc700 1 --2- 172.21.15.123:0/1274686471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80c43a0 0x7fe5d80c69d0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fe5d80c8900 tx=0x7fe5d0009780 comp rx=0 tx=0).stop 2023-10-06T04:31:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fa89c82a700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa898156fe0 0x7fa8980ad4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56906/0 (socket says 172.21.15.123:56906) 2023-10-06T04:31:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fa89c82a700 1 -- 172.21.15.123:0/3951570804 learned_addr learned my addr 172.21.15.123:0/3951570804 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:40.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7fa897fff700 1 --2- 172.21.15.123:0/3951570804 >> v2:172.21.15.177:3300/0 conn(0x7fa8980adbc0 0x7fa8980c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.956+0000 7f807546f700 1 -- start start 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7f807546f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 0x7f8070156910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7f807546f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80701592b0 0x7f80700c82d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7f807546f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8070159680 0x7f8070176f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7f807546f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f80700b4a90 con 0x7f80701592b0 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7f807546f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8070157b20 con 0x7f80700acea0 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.957+0000 7f807546f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f80700c76c0 con 0x7f8070159680 2023-10-06T04:31:40.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80701592b0 0x7f80700c82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80701592b0 0x7f80700c82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43154/0 (socket says 172.21.15.123:43154) 2023-10-06T04:31:40.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 -- 172.21.15.123:0/1456244899 learned_addr learned my addr 172.21.15.123:0/1456244899 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806effd700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 0x7f8070156910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806f7fe700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8070159680 0x7f8070176f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fe5ddacc700 1 -- 172.21.15.123:0/1274686471 shutdown_connections 2023-10-06T04:31:40.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fe5ddacc700 1 --2- 172.21.15.123:0/1274686471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80c43a0 0x7fe5d80c69d0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fe5ddacc700 1 --2- 172.21.15.123:0/1274686471 >> v2:172.21.15.123:3300/0 conn(0x7fe5d80698c0 0x7fe5d80c6490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fe5ddacc700 1 -- 172.21.15.123:0/1274686471 >> 172.21.15.123:0/1274686471 conn(0x7fe5d80680f0 msgr2=0x7fe5d80cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fa8977fe700 1 -- 172.21.15.123:0/3951570804 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1455361736 0 0) 0x7fa8980ae050 con 0x7fa898156fe0 2023-10-06T04:31:40.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fa8977fe700 1 -- 172.21.15.123:0/3951570804 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa880003570 con 0x7fa898156fe0 2023-10-06T04:31:40.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fa897fff700 1 -- 172.21.15.123:0/3951570804 >> v1:172.21.15.177:6789/0 conn(0x7fa8981573b0 legacy=0x7fa898069730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fa897fff700 1 -- 172.21.15.123:0/3951570804 >> v1:172.21.15.123:6789/0 conn(0x7fa898156fe0 legacy=0x7fa8980ad4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fa897fff700 1 -- 172.21.15.123:0/3951570804 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8980ae610 con 0x7fa8980adbc0 2023-10-06T04:31:40.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fe5ddacc700 1 -- 172.21.15.123:0/1274686471 shutdown_connections 2023-10-06T04:31:40.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fa897fff700 1 --2- 172.21.15.123:0/3951570804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 0x7fa8980c88b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009860 tx=0x7fa88c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3fc3b81315c5f4d8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 -- 172.21.15.123:0/1456244899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8070159680 msgr2=0x7f8070176f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8070159680 0x7f8070176f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 -- 172.21.15.123:0/1456244899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 msgr2=0x7f8070156910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f80700acea0 0x7f8070156910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7f806e7fc700 1 -- 172.21.15.123:0/1456244899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80600094f0 con 0x7f80701592b0 2023-10-06T04:31:40.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fa8977fe700 1 -- 172.21.15.123:0/3951570804 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c011630 con 0x7fa8980adbc0 2023-10-06T04:31:40.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7fa8977fe700 1 -- 172.21.15.123:0/3951570804 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa88c011790 con 0x7fa8980adbc0 2023-10-06T04:31:40.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7fa8977fe700 1 -- 172.21.15.123:0/3951570804 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88c011b30 con 0x7fa8980adbc0 2023-10-06T04:31:40.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7f806e7fc700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80701592b0 0x7f80700c82d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8070064840 tx=0x7f8060016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.958+0000 7fe5ddacc700 1 -- 172.21.15.123:0/1274686471 wait complete. 2023-10-06T04:31:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7fa89ea8e700 1 -- 172.21.15.123:0/3951570804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 msgr2=0x7fa8980c88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7fa89ea8e700 1 --2- 172.21.15.123:0/3951570804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 0x7fa8980c88b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fa88c009860 tx=0x7fa88c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fa89ea8e700 1 -- 172.21.15.123:0/3951570804 shutdown_connections 2023-10-06T04:31:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fa89ea8e700 1 --2- 172.21.15.123:0/3951570804 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 0x7fa8980c88b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fa89ea8e700 1 -- 172.21.15.123:0/3951570804 >> 172.21.15.123:0/3951570804 conn(0x7fa898067660 msgr2=0x7fa898067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6e8c01a400 con 0x7f6e9c06f180 2023-10-06T04:31:40.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7f6e917fa700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6e80067000 0x7f6e800694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6e8c084630 con 0x7f6e9c06f180 2023-10-06T04:31:40.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7f6e917fa700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f6e8006f8b0 0x7f6e80071cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6e80072370 con 0x7f6e8006f8b0 2023-10-06T04:31:40.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6e8c084960 con 0x7f6e9c06f180 2023-10-06T04:31:40.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fe5ddacc700 1 Processor -- start 2023-10-06T04:31:40.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fa89ea8e700 1 -- 172.21.15.123:0/3951570804 shutdown_connections 2023-10-06T04:31:40.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7fd949a30700 1 Processor -- start 2023-10-06T04:31:40.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7fd949a30700 1 -- start start 2023-10-06T04:31:40.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fa89ea8e700 1 -- 172.21.15.123:0/3951570804 wait complete. 2023-10-06T04:31:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7f6ea0fb4700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6e80067000 0x7f6e800694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7f6ea0fb4700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6e80067000 0x7f6e800694b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c161a70 tx=0x7f6e980090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060011d40 con 0x7f80701592b0 2023-10-06T04:31:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8060020070 con 0x7f80701592b0 2023-10-06T04:31:40.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.959+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8060011ea0 con 0x7f80701592b0 2023-10-06T04:31:40.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8070156f10 con 0x7f80701592b0 2023-10-06T04:31:40.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f80701574d0 con 0x7f80701592b0 2023-10-06T04:31:40.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7f6ea1fb6700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f6e8006f8b0 0x7f6e80071cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7f6ea1fb6700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f6e8006f8b0 0x7f6e80071cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fe5ddacc700 1 -- start start 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fe5ddacc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80698c0 0x7fe5d80b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fe5ddacc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5d80bf330 0x7fe5d814f0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fe5ddacc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d80c43a0 0x7fe5d80c9d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fe5ddacc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80b1420 con 0x7fe5d80bf330 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fe5ddacc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80b0810 con 0x7fe5d80698c0 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fe5ddacc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d80b0af0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80698c0 0x7fe5d80b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d7fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d80c43a0 0x7fe5d80c9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7fa89ea8e700 1 Processor -- start 2023-10-06T04:31:40.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fa89ea8e700 1 -- start start 2023-10-06T04:31:40.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.961+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f807006a1f0 con 0x7f80701592b0 2023-10-06T04:31:40.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d77fe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80698c0 0x7fe5d80b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41004/0 (socket says 172.21.15.123:41004) 2023-10-06T04:31:40.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d77fe700 1 -- 172.21.15.123:0/3950464250 learned_addr learned my addr 172.21.15.123:0/3950464250 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5ceffd700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5d80bf330 0x7fe5d814f0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d7fff700 1 -- 172.21.15.123:0/3950464250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80698c0 msgr2=0x7fe5d80b18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d7fff700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe5d80698c0 0x7fe5d80b18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d7fff700 1 -- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5d80bf330 msgr2=0x7fe5d814f0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d7fff700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe5d80bf330 0x7fe5d814f0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d7fff700 1 -- 172.21.15.123:0/3950464250 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d0009a20 con 0x7fe5d80c43a0 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fe5d7fff700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d80c43a0 0x7fe5d80c9d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0027000 tx=0x7fe5d002a000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fa89ea8e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 0x7fa89815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fa89ea8e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898156fe0 0x7fa8980c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fa89ea8e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8981573b0 0x7fa89806d4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fa89ea8e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8981587a0 con 0x7fa898156fe0 2023-10-06T04:31:40.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fa89ea8e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa89814a3b0 con 0x7fa8980adbc0 2023-10-06T04:31:40.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fa89ea8e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa898171720 con 0x7fa8981573b0 2023-10-06T04:31:40.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6e80072370 con 0x7f6e8006f8b0 2023-10-06T04:31:40.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.960+0000 7fd949a30700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fd942ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.962+0000 7fd942ffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440691b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46372/0 (socket says 172.21.15.123:46372) 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd949a30700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd9440696f0 0x7fd9440c74a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd942ffd700 1 -- 172.21.15.123:0/3564428630 learned_addr learned my addr 172.21.15.123:0/3564428630 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd944069b50 con 0x7fd9440cca10 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89c82a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 0x7fa89815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89d02b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8981573b0 0x7fa89806d4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89c82a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 0x7fa89815a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41018/0 (socket says 172.21.15.123:41018) 2023-10-06T04:31:40.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89c82a700 1 -- 172.21.15.123:0/2574460666 learned_addr learned my addr 172.21.15.123:0/2574460666 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9440c2f00 con 0x7fd9440696f0 2023-10-06T04:31:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440c7cf0 con 0x7fd9440bf330 2023-10-06T04:31:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd9437fe700 1 --2- 172.21.15.123:0/3564428630 >> v2:172.21.15.123:3300/0 conn(0x7fd9440696f0 0x7fd9440c74a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa897fff700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898156fe0 0x7fa8980c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa6988bb700 1 Processor -- start 2023-10-06T04:31:40.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa6988bb700 1 -- start start 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd941ffb700 1 -- 172.21.15.123:0/3564428630 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723949891 0 0) 0x7fd938001110 con 0x7fd9440cca10 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd941ffb700 1 -- 172.21.15.123:0/3564428630 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd92c003570 con 0x7fd9440cca10 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd942ffd700 1 -- 172.21.15.123:0/3564428630 >> v2:172.21.15.123:3300/0 conn(0x7fd9440696f0 msgr2=0x7fd9440c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd942ffd700 1 --2- 172.21.15.123:0/3564428630 >> v2:172.21.15.123:3300/0 conn(0x7fd9440696f0 0x7fd9440c74a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd942ffd700 1 -- 172.21.15.123:0/3564428630 >> v1:172.21.15.177:6789/0 conn(0x7fd9440cca10 legacy=0x7fd9440c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd942ffd700 1 -- 172.21.15.123:0/3564428630 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9440c7fd0 con 0x7fd9440bf330 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fd942ffd700 1 --2- 172.21.15.123:0/3564428630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440691b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9340099b0 tx=0x7fd934009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5840321edc35bb15 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89d02b700 1 -- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 msgr2=0x7fa89815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89d02b700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8980adbc0 0x7fa89815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89d02b700 1 -- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898156fe0 msgr2=0x7fa8980c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89d02b700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa898156fe0 0x7fa8980c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89d02b700 1 -- 172.21.15.123:0/2574460666 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa88c009420 con 0x7fa8981573b0 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fa89d02b700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8981573b0 0x7fa89806d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c5b0 tx=0x7fa888007de0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.964+0000 7fa6988bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa694156fe0 0x7fa6940ad4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.964+0000 7fa6988bb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa6941573b0 0x7fa6940ad9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.964+0000 7fa6988bb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6940ae050 con 0x7fa6940c88b0 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.964+0000 7fa6988bb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa6940692e0 con 0x7fa6941573b0 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.964+0000 7fa6988bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940ae330 con 0x7fa694156fe0 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa69259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa694156fe0 0x7fa6940ad4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa69259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa694156fe0 0x7fa6940ad4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46392/0 (socket says 172.21.15.123:46392) 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa69259c700 1 -- 172.21.15.123:0/1837806890 learned_addr learned my addr 172.21.15.123:0/1837806890 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa69259c700 1 -- 172.21.15.123:0/1837806890 >> v2:172.21.15.123:3300/0 conn(0x7fa6941573b0 msgr2=0x7fa6940ad9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa69259c700 1 --2- 172.21.15.123:0/1837806890 >> v2:172.21.15.123:3300/0 conn(0x7fa6941573b0 0x7fa6940ad9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa69259c700 1 -- 172.21.15.123:0/1837806890 >> v1:172.21.15.123:6789/0 conn(0x7fa6940c88b0 legacy=0x7fa69406d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa69259c700 1 -- 172.21.15.123:0/1837806890 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6940ae610 con 0x7fa694156fe0 2023-10-06T04:31:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.964+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8060022030 con 0x7f80701592b0 2023-10-06T04:31:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7f8056ffd700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f805c067000 0x7f805c0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7f806effd700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f805c067000 0x7f805c0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d002e630 con 0x7fe5d80c43a0 2023-10-06T04:31:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5d002e7b0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.963+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5d002eb70 con 0x7fe5d80c43a0 2023-10-06T04:31:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d80b0dd0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d80c0e80 con 0x7fe5d80c43a0 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fd941ffb700 1 -- 172.21.15.123:0/3564428630 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400b680 con 0x7fd9440bf330 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fd941ffb700 1 -- 172.21.15.123:0/3564428630 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93400b7e0 con 0x7fd9440bf330 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fd941ffb700 1 -- 172.21.15.123:0/3564428630 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400bb80 con 0x7fd9440bf330 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 msgr2=0x7fd9440691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fd949a30700 1 --2- 172.21.15.123:0/3564428630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440691b0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fd9340099b0 tx=0x7fd934009250 comp rx=0 tx=0).stop 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 shutdown_connections 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fd949a30700 1 --2- 172.21.15.123:0/3564428630 >> v2:172.21.15.123:3300/0 conn(0x7fd9440696f0 0x7fd9440c74a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fd949a30700 1 --2- 172.21.15.123:0/3564428630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440691b0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 >> 172.21.15.123:0/3564428630 conn(0x7fd9440680f0 msgr2=0x7fd9440cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa88800cea0 con 0x7fa8981573b0 2023-10-06T04:31:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa888011590 con 0x7fa8981573b0 2023-10-06T04:31:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa888011970 con 0x7fa8981573b0 2023-10-06T04:31:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa898171a00 con 0x7fa8981573b0 2023-10-06T04:31:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.965+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa898171fc0 con 0x7fa8981573b0 2023-10-06T04:31:40.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fa69259c700 1 --2- 172.21.15.123:0/1837806890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa694156fe0 0x7fa6940ad4b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6880099b0 tx=0x7fa688009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=845d3814f6a74863 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 shutdown_connections 2023-10-06T04:31:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fd949a30700 1 -- 172.21.15.123:0/3564428630 wait complete. 2023-10-06T04:31:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fa69159a700 1 -- 172.21.15.123:0/1837806890 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800b820 con 0x7fa694156fe0 2023-10-06T04:31:40.993 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fe5bc000fd0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fa69159a700 1 -- 172.21.15.123:0/1837806890 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68800b980 con 0x7fa694156fe0 2023-10-06T04:31:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7fa69159a700 1 -- 172.21.15.123:0/1837806890 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68800bd20 con 0x7fa694156fe0 2023-10-06T04:31:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.966+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f80600864d0 con 0x7f80701592b0 2023-10-06T04:31:40.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7f8056ffd700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f805c06f830 0x7f805c071c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f805c0722f0 con 0x7f805c06f830 2023-10-06T04:31:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7f806effd700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f805c067000 0x7f805c0694b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8064009b40 tx=0x7f80640090f0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f8060053660 con 0x7f80701592b0 2023-10-06T04:31:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7fa6988bb700 1 -- 172.21.15.123:0/1837806890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa694156fe0 msgr2=0x7fa6940ad4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7fa6988bb700 1 --2- 172.21.15.123:0/1837806890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa694156fe0 0x7fa6940ad4b0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fa6880099b0 tx=0x7fa688009250 comp rx=0 tx=0).stop 2023-10-06T04:31:40.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7fd949a30700 1 Processor -- start 2023-10-06T04:31:40.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7fd949a30700 1 -- start start 2023-10-06T04:31:40.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7f806f7fe700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f805c06f830 0x7f805c071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7f806f7fe700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f805c06f830 0x7f805c071c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:40.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.967+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe5d0035070 con 0x7fe5d80c43a0 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fe5cffff700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe5b8066f80 0x7fe5b8069430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe5d00985d0 con 0x7fe5d80c43a0 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fe5cffff700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7fe5b806f830 0x7fe5b8071c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fe5b80722f0 con 0x7fe5b806f830 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fa6988bb700 1 -- 172.21.15.123:0/1837806890 shutdown_connections 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fa6988bb700 1 --2- 172.21.15.123:0/1837806890 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa694156fe0 0x7fa6940ad4b0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fa6988bb700 1 --2- 172.21.15.123:0/1837806890 >> v2:172.21.15.123:3300/0 conn(0x7fa6941573b0 0x7fa6940ad9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:40.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fa6988bb700 1 -- 172.21.15.123:0/1837806890 >> 172.21.15.123:0/1837806890 conn(0x7fa694067660 msgr2=0x7fa694067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd949a30700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9440696f0 0x7fd9440c1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd949a30700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440af740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd949a30700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9440cca10 0x7fd9440afc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd949a30700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440c0d50 con 0x7fd9440696f0 2023-10-06T04:31:40.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe5d0098900 con 0x7fe5d80c43a0 2023-10-06T04:31:40.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd949a30700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440b0300 con 0x7fd9440cca10 2023-10-06T04:31:40.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd949a30700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9440c18b0 con 0x7fd9440bf330 2023-10-06T04:31:40.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd942ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9440696f0 0x7fd9440c1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd942ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9440696f0 0x7fd9440c1d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43186/0 (socket says 172.21.15.123:43186) 2023-10-06T04:31:40.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd942ffd700 1 -- 172.21.15.123:0/2363718796 learned_addr learned my addr 172.21.15.123:0/2363718796 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:40.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd9427fc700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440af740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:40.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fa6988bb700 1 -- 172.21.15.123:0/1837806890 shutdown_connections 2023-10-06T04:31:41.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fd9437fe700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9440cca10 0x7fd9440afc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.968+0000 7fe5d77fe700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe5b8066f80 0x7fe5b8069430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== osd.2 v2:172.21.15.123:6824/119723 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f805c0722f0 con 0x7f805c06f830 2023-10-06T04:31:41.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7fe5d77fe700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe5b8066f80 0x7fe5b8069430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0007130 tx=0x7fe5c0007f70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7fa6988bb700 1 -- 172.21.15.123:0/1837806890 wait complete. 2023-10-06T04:31:41.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.970+0000 7fa6988bb700 1 Processor -- start 2023-10-06T04:31:41.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.970+0000 7fa6988bb700 1 -- start start 2023-10-06T04:31:41.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.971+0000 7fe5ceffd700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7fe5b806f830 0x7fe5b8071c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.971+0000 7fe5ceffd700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7fe5b806f830 0x7fe5b8071c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7fd942ffd700 1 -- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 msgr2=0x7fd9440af740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7fd942ffd700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd9440bf330 0x7fd9440af740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7fd942ffd700 1 -- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9440cca10 msgr2=0x7fd9440afc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7fd942ffd700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd9440cca10 0x7fd9440afc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.969+0000 7fd942ffd700 1 -- 172.21.15.123:0/2363718796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9340094f0 con 0x7fd9440696f0 2023-10-06T04:31:41.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.970+0000 7fd942ffd700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9440696f0 0x7fd9440c1d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd94406fae0 tx=0x7fd934017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.970+0000 7fd92affd700 1 -- 172.21.15.123:0/2363718796 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93400b950 con 0x7fd9440696f0 2023-10-06T04:31:41.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.970+0000 7fd92affd700 1 -- 172.21.15.123:0/2363718796 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd93400bad0 con 0x7fd9440696f0 2023-10-06T04:31:41.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.970+0000 7fd92affd700 1 -- 172.21.15.123:0/2363718796 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd93401a680 con 0x7fd9440696f0 2023-10-06T04:31:41.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.970+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9440b1a60 con 0x7fd9440696f0 2023-10-06T04:31:41.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.971+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9440c0f80 con 0x7fd9440696f0 2023-10-06T04:31:41.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.971+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa89806c3d0 con 0x7fa8981573b0 2023-10-06T04:31:41.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7f6d0d074700 1 Processor -- start 2023-10-06T04:31:41.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa6988bb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940c88b0 0x7fa6941577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7f40d7e03700 1 Processor -- start 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa6988bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa694156fe0 0x7fa694154ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa6988bb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6941573b0 0x7fa6941555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa6988bb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c4950 con 0x7fa694156fe0 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa6988bb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa694157e80 con 0x7fa6941573b0 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa6988bb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6940c4020 con 0x7fa6940c88b0 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa691d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa694156fe0 0x7fa694154ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa691d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa694156fe0 0x7fa694154ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43192/0 (socket says 172.21.15.123:43192) 2023-10-06T04:31:41.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7fa691d9b700 1 -- 172.21.15.123:0/1968431105 learned_addr learned my addr 172.21.15.123:0/1968431105 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7f40d7e03700 1 -- start start 2023-10-06T04:31:41.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa691d9b700 1 -- 172.21.15.123:0/1968431105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940c88b0 msgr2=0x7fa6941577f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:31:41.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa691d9b700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa6940c88b0 0x7fa6941577f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa691d9b700 1 -- 172.21.15.123:0/1968431105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6941573b0 msgr2=0x7fa6941555b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:31:41.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa691d9b700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa6941573b0 0x7fa6941555b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa691d9b700 1 -- 172.21.15.123:0/1968431105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6880094f0 con 0x7fa694156fe0 2023-10-06T04:31:41.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.972+0000 7f6d0d074700 1 -- start start 2023-10-06T04:31:41.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7f6d0d074700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6d0815d6f0 0x7f6d0806e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7f6d0d074700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d08162060 con 0x7f6d0806e990 2023-10-06T04:31:41.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7f6d0d074700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d08161450 con 0x7f6d08068d80 2023-10-06T04:31:41.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7f6d0d074700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d08161730 con 0x7f6d0815d6f0 2023-10-06T04:31:41.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa691d9b700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa694156fe0 0x7fa694154ec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00bb60 tx=0x7fa68c009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7f6d0659c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6d0806e990 0x7f6d08068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56932/0 (socket says 172.21.15.123:56932) 2023-10-06T04:31:41.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7f6d0659c700 1 -- 172.21.15.123:0/786769766 learned_addr learned my addr 172.21.15.123:0/786769766 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:41.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00c940 con 0x7fa694156fe0 2023-10-06T04:31:41.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f6d06d9d700 1 --2- 172.21.15.123:0/786769766 >> v2:172.21.15.177:3300/0 conn(0x7f6d0815d6f0 0x7f6d0806e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa68c00cac0 con 0x7fa694156fe0 2023-10-06T04:31:41.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa68c00ce80 con 0x7fa694156fe0 2023-10-06T04:31:41.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6940c4300 con 0x7fa694156fe0 2023-10-06T04:31:41.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6940c4de0 con 0x7fa694156fe0 2023-10-06T04:31:41.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fa69406c3d0 con 0x7fa694156fe0 2023-10-06T04:31:41.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fe5b80722f0 con 0x7fe5b806f830 2023-10-06T04:31:41.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d7e03700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40d00688c0 0x7f40d006a5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d7e03700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f40d0155b40 0x7f40d00cb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d7e03700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d006ab10 con 0x7f40d0068c90 2023-10-06T04:31:41.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d7e03700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d0155f50 con 0x7f40d0155b40 2023-10-06T04:31:41.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d7e03700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d00c6dd0 con 0x7f40d00688c0 2023-10-06T04:31:41.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d5b9f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40d00688c0 0x7f40d006a5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d5b9f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f40d00688c0 0x7f40d006a5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41062/0 (socket says 172.21.15.123:41062) 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d5b9f700 1 -- 172.21.15.123:0/3946831969 learned_addr learned my addr 172.21.15.123:0/3946831969 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f40d63a0700 1 --2- 172.21.15.123:0/3946831969 >> v2:172.21.15.123:3300/0 conn(0x7f40d0155b40 0x7f40d00cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f40d5b9f700 1 -- 172.21.15.123:0/3946831969 >> v2:172.21.15.123:3300/0 conn(0x7f40d0155b40 msgr2=0x7f40d00cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f40d5b9f700 1 --2- 172.21.15.123:0/3946831969 >> v2:172.21.15.123:3300/0 conn(0x7f40d0155b40 0x7f40d00cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f40d5b9f700 1 -- 172.21.15.123:0/3946831969 >> v1:172.21.15.177:6789/0 conn(0x7f40d0068c90 legacy=0x7f40d0155430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f40d5b9f700 1 -- 172.21.15.123:0/3946831969 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00cc3e0 con 0x7f40d00688c0 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f6d06d9d700 1 -- 172.21.15.123:0/786769766 >> v1:172.21.15.177:6789/0 conn(0x7f6d08068d80 legacy=0x7f6d0815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f6d06d9d700 1 -- 172.21.15.123:0/786769766 >> v1:172.21.15.123:6789/0 conn(0x7f6d0806e990 legacy=0x7f6d08068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f6d06d9d700 1 -- 172.21.15.123:0/786769766 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d08161a10 con 0x7f6d0815d6f0 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7f6d06d9d700 1 --2- 172.21.15.123:0/786769766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0815d6f0 0x7f6d0806e450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80098e0 tx=0x7f6cf80091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b9f08839d54c29d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f6d05d9b700 1 -- 172.21.15.123:0/786769766 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf8009e80 con 0x7f6d0815d6f0 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f6d05d9b700 1 -- 172.21.15.123:0/786769766 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cf800b760 con 0x7f6d0815d6f0 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f6d05d9b700 1 -- 172.21.15.123:0/786769766 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cf800bb00 con 0x7f6d0815d6f0 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.973+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa88801c440 con 0x7fa8981573b0 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa894ff9700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa884067080 0x7fa884069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa888083f20 con 0x7fa8981573b0 2023-10-06T04:31:41.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa894ff9700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7fa88406f930 0x7fa884071d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa8840723f0 con 0x7fa88406f930 2023-10-06T04:31:41.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa888084250 con 0x7fa8981573b0 2023-10-06T04:31:41.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa897fff700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7fa88406f930 0x7fa884071d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.974+0000 7fa89c82a700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa884067080 0x7fa884069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7f40d5b9f700 1 --2- 172.21.15.123:0/3946831969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d00688c0 0x7f40d006a5d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40d006f3b0 tx=0x7f40cc009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfea0590fa824515 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7fa897fff700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7fa88406f930 0x7fa884071d40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.975+0000 7fa89c82a700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa884067080 0x7fa884069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa890002250 tx=0x7fa890001ae0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa68c014620 con 0x7fa694156fe0 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.977+0000 7fa6827fc700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa674066f30 0x7fa6740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.977+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa68c0852f0 con 0x7fa694156fe0 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.977+0000 7fa69259c700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa674066f30 0x7fa6740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.977+0000 7fa6827fc700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7fa67406f7e0 0x7fa674071bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.977+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fa6740722a0 con 0x7fa67406f7e0 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.977+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7fa68c0855e0 con 0x7fa694156fe0 2023-10-06T04:31:41.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7fa69259c700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa674066f30 0x7fa6740693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6880099b0 tx=0x7fa688016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.980+0000 7f40d4b9d700 1 -- 172.21.15.123:0/3946831969 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc009e70 con 0x7f40d00688c0 2023-10-06T04:31:41.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.980+0000 7f40d4b9d700 1 -- 172.21.15.123:0/3946831969 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc00b860 con 0x7f40d00688c0 2023-10-06T04:31:41.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.980+0000 7f40d4b9d700 1 -- 172.21.15.123:0/3946831969 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc00bc00 con 0x7f40d00688c0 2023-10-06T04:31:41.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7f6d0d074700 1 -- 172.21.15.123:0/786769766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0815d6f0 msgr2=0x7f6d0806e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7f6d0d074700 1 --2- 172.21.15.123:0/786769766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0815d6f0 0x7f6d0806e450 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80098e0 tx=0x7f6cf80091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7f6d0d074700 1 -- 172.21.15.123:0/786769766 shutdown_connections 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7f6d0d074700 1 --2- 172.21.15.123:0/786769766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d0815d6f0 0x7f6d0806e450 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7f6d0d074700 1 -- 172.21.15.123:0/786769766 >> 172.21.15.123:0/786769766 conn(0x7f6d080c9d00 msgr2=0x7f6d0806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7f6d0d074700 1 -- 172.21.15.123:0/786769766 shutdown_connections 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.976+0000 7f6d0d074700 1 -- 172.21.15.123:0/786769766 wait complete. 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 Processor -- start 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 -- start start 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d08068d80 0x7f6d080c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d0806e990 0x7f6d0814f9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0815d6f0 0x7f6d0806b580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d080c4bc0 con 0x7f6d0806e990 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0806a4e0 con 0x7f6d08068d80 2023-10-06T04:31:41.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.978+0000 7f6d0d074700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d0806a7c0 con 0x7f6d0815d6f0 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d0806e990 0x7f6d0814f9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d0806e990 0x7f6d0814f9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43210/0 (socket says 172.21.15.123:43210) 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 -- 172.21.15.123:0/2095273333 learned_addr learned my addr 172.21.15.123:0/2095273333 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d06d9d700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d08068d80 0x7f6d080c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0759e700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0815d6f0 0x7f6d0806b580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 -- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0815d6f0 msgr2=0x7f6d0806b580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0815d6f0 0x7f6d0806b580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 -- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d08068d80 msgr2=0x7f6d080c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6d08068d80 0x7f6d080c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 -- 172.21.15.123:0/2095273333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cf8009420 con 0x7f6d0806e990 2023-10-06T04:31:41.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0759e700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6d0815d6f0 0x7f6d0806b580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:41.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6d0659c700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d0806e990 0x7f6d0814f9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c7e0 tx=0x7f6cfc007e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc005b30 con 0x7f6d0806e990 2023-10-06T04:31:41.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cfc005cb0 con 0x7f6d0806e990 2023-10-06T04:31:41.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.979+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cfc007320 con 0x7f6d0806e990 2023-10-06T04:31:41.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.980+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d0806aaa0 con 0x7f6d0806e990 2023-10-06T04:31:41.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.980+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d0806b060 con 0x7f6d0806e990 2023-10-06T04:31:41.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.983+0000 7fa692d9d700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7fa67406f7e0 0x7fa674071bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.983+0000 7f40d7e03700 1 -- 172.21.15.123:0/3946831969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d00688c0 msgr2=0x7f40d006a5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.983+0000 7f40d7e03700 1 --2- 172.21.15.123:0/3946831969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d00688c0 0x7f40d006a5d0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40d006f3b0 tx=0x7f40cc009250 comp rx=0 tx=0).stop 2023-10-06T04:31:41.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.980+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== osd.7 v2:172.21.15.177:6824/119977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa8840723f0 con 0x7fa88406f930 2023-10-06T04:31:41.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.984+0000 7fa692d9d700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7fa67406f7e0 0x7fa674071bf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fa6740722a0 con 0x7fa67406f7e0 2023-10-06T04:31:41.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f40d7e03700 1 -- 172.21.15.123:0/3946831969 shutdown_connections 2023-10-06T04:31:41.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f40d7e03700 1 --2- 172.21.15.123:0/3946831969 >> v2:172.21.15.123:3300/0 conn(0x7f40d0155b40 0x7f40d00cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f40d7e03700 1 --2- 172.21.15.123:0/3946831969 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d00688c0 0x7f40d006a5d0 secure :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f40d006f3b0 tx=0x7f40cc009250 comp rx=0 tx=0).stop 2023-10-06T04:31:41.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f40d7e03700 1 -- 172.21.15.123:0/3946831969 >> 172.21.15.123:0/3946831969 conn(0x7f40d00c9d00 msgr2=0x7f40d006da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f6d0806c9b0 con 0x7f6d0806e990 2023-10-06T04:31:41.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6cfc01b3b0 con 0x7f6d0806e990 2023-10-06T04:31:41.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f6ceeffd700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6cf4066f30 0x7f6cf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6cfc083b40 con 0x7f6d0806e990 2023-10-06T04:31:41.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f6ceeffd700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f6cf406f760 0x7f6cf4071b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f6cf4072220 con 0x7f6cf406f760 2023-10-06T04:31:41.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.985+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6cfc050c50 con 0x7f6d0806e990 2023-10-06T04:31:41.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.986+0000 7f6d0759e700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f6cf406f760 0x7f6cf4071b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.986+0000 7f6d0759e700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f6cf406f760 0x7f6cf4071b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.987+0000 7f6d06d9d700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6cf4066f30 0x7f6cf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.987+0000 7f6d06d9d700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6cf4066f30 0x7f6cf40693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80030c0 tx=0x7f6cf8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.989+0000 7f40d7e03700 1 -- 172.21.15.123:0/3946831969 shutdown_connections 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.990+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6e9c1615b0 con 0x7f6e8006f8b0 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.987+0000 7fd92affd700 1 -- 172.21.15.123:0/2363718796 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd93401aa30 con 0x7fd9440696f0 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.987+0000 7fd92affd700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd92c0673b0 0x7fd92c069860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.988+0000 7fd92affd700 1 -- 172.21.15.123:0/2363718796 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd9340862c0 con 0x7fd9440696f0 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.988+0000 7fd9427fc700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd92c0673b0 0x7fd92c069860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.988+0000 7fd949a30700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7fd9240015f0 0x7fd924003aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.988+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fd924006b90 con 0x7fd9240015f0 2023-10-06T04:31:41.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.988+0000 7fd9427fc700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd92c0673b0 0x7fd92c069860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9440c3ee0 tx=0x7fd938007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.990+0000 7f6e917fa700 1 -- 172.21.15.123:0/3856206091 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6e9c1615b0 con 0x7f6e8006f8b0 2023-10-06T04:31:41.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.990+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== osd.5 v2:172.21.15.177:6800/119947 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f6cf4072220 con 0x7f6cf406f760 2023-10-06T04:31:41.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.992+0000 7f40d7e03700 1 -- 172.21.15.123:0/3946831969 wait complete. 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.994+0000 7f40d7e03700 1 Processor -- start 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.994+0000 7f40d7e03700 1 -- start start 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d7e03700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0068c90 0x7f40d00abca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d7e03700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d0155b40 0x7f40d014a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d7e03700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d00c1620 0x7f40d00c1ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d7e03700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00c4830 con 0x7f40d00c1620 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d7e03700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d00c2000 con 0x7f40d0068c90 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d7e03700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d014a8d0 con 0x7f40d0155b40 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d539e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d0155b40 0x7f40d014a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d539e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d0155b40 0x7f40d014a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46414/0 (socket says 172.21.15.123:46414) 2023-10-06T04:31:41.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d539e700 1 -- 172.21.15.123:0/3175826279 learned_addr learned my addr 172.21.15.123:0/3175826279 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.995+0000 7f40d63a0700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d00c1620 0x7f40d00c1ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d5b9f700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0068c90 0x7f40d00abca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d539e700 1 -- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0068c90 msgr2=0x7f40d00abca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d539e700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f40d0068c90 0x7f40d00abca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d539e700 1 -- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d00c1620 msgr2=0x7f40d00c1ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d539e700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f40d00c1620 0x7f40d00c1ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d539e700 1 -- 172.21.15.123:0/3175826279 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40cc0094f0 con 0x7f40d0155b40 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d539e700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d0155b40 0x7f40d014a390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c590 tx=0x7f40c000c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0011590 con 0x7f40d0155b40 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.996+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d00c4550 con 0x7f40d0155b40 2023-10-06T04:31:41.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d00c52c0 con 0x7f40d0155b40 2023-10-06T04:31:41.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7fd9437fe700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7fd9240015f0 0x7fd924003aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.992+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f6e8006f8b0 msgr2=0x7f6e80071cc0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.992+0000 7f6ea3a19700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f6e8006f8b0 0x7f6e80071cc0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6e80067000 msgr2=0x7f6e800694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f6ea3a19700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6e80067000 0x7f6e800694b0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f6e9c161a70 tx=0x7f6e980090f0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c06f180 msgr2=0x7f6e9c0cc3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f6ea3a19700 1 --2- 172.21.15.123:0/3856206091 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6e9c06f180 0x7f6e9c0cc3f0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6e8c00b810 tx=0x7f6e8c009250 comp rx=0 tx=0).stop 2023-10-06T04:31:41.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f6ea1fb6700 1 -- 172.21.15.123:0/3856206091 reap_dead start 2023-10-06T04:31:41.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 shutdown_connections 2023-10-06T04:31:41.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 >> 172.21.15.123:0/3856206091 conn(0x7f6e9c067660 msgr2=0x7f6e9c0ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40c00116f0 con 0x7f40d0155b40 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.997+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40c0011a90 con 0x7f40d0155b40 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.998+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40b0000fd0 con 0x7f40d0155b40 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.998+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40c001b3b0 con 0x7f40d0155b40 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.999+0000 7f40c5ffb700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40b4066f60 0x7f40b4069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.999+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f40c0083b90 con 0x7f40d0155b40 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.999+0000 7f40c5ffb700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f40b406f810 0x7f40b4071c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.999+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40b40722d0 con 0x7f40b406f810 2023-10-06T04:31:41.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.999+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_get_version_reply(handle=1 version=23) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40c0088220 con 0x7f40d0155b40 2023-10-06T04:31:41.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:40.999+0000 7f40d63a0700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f40b406f810 0x7f40b4071c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.000+0000 7f40d63a0700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f40b406f810 0x7f40b4071c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.000+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f80700ca870 con 0x7f805c06f830 2023-10-06T04:31:41.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.000+0000 7f8056ffd700 1 -- 172.21.15.123:0/1456244899 <== osd.2 v2:172.21.15.123:6824/119723 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f80700ca870 con 0x7f805c06f830 2023-10-06T04:31:41.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.000+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== osd.4 v2:172.21.15.177:6808/119946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f40b40722d0 con 0x7f40b406f810 2023-10-06T04:31:41.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.000+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa8981721b0 con 0x7fa88406f930 2023-10-06T04:31:41.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.000+0000 7fa894ff9700 1 -- 172.21.15.123:0/2574460666 <== osd.7 v2:172.21.15.177:6824/119977 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa8981721b0 con 0x7fa88406f930 2023-10-06T04:31:41.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.001+0000 7f40d5b9f700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40b4066f60 0x7f40b4069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.001+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7fa88406f930 msgr2=0x7fa884071d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.001+0000 7fa89ea8e700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7fa88406f930 0x7fa884071d40 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f40d5b9f700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40b4066f60 0x7f40b4069410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40d006f3b0 tx=0x7f40cc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa884067080 msgr2=0x7fa884069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fa89ea8e700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa884067080 0x7fa884069530 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fa890002250 tx=0x7fa890001ae0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8981573b0 msgr2=0x7fa89806d4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fa89ea8e700 1 --2- 172.21.15.123:0/2574460666 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8981573b0 0x7fa89806d4e0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fa88800c5b0 tx=0x7fa888007de0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fa89d02b700 1 -- 172.21.15.123:0/2574460666 reap_dead start 2023-10-06T04:31:41.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 shutdown_connections 2023-10-06T04:31:41.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 >> 172.21.15.123:0/2574460666 conn(0x7fa898067660 msgr2=0x7fa8980c8700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.035 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f805c06f830 msgr2=0x7f805c071c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.035 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f807546f700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f805c06f830 0x7f805c071c40 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.035 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 shutdown_connections 2023-10-06T04:31:41.035 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f6ea3a19700 1 -- 172.21.15.123:0/3856206091 wait complete. 2023-10-06T04:31:41.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f805c067000 msgr2=0x7f805c0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f807546f700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f805c067000 0x7f805c0694b0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f8064009b40 tx=0x7f80640090f0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80701592b0 msgr2=0x7f80700c82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7f807546f700 1 --2- 172.21.15.123:0/1456244899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f80701592b0 0x7f80700c82d0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f8070064840 tx=0x7f8060016000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.010+0000 7f806f7fe700 1 -- 172.21.15.123:0/1456244899 reap_dead start 2023-10-06T04:31:41.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.010+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 shutdown_connections 2023-10-06T04:31:41.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.010+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 >> 172.21.15.123:0/1456244899 conn(0x7f80700995a0 msgr2=0x7f80700cb8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.010+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 shutdown_connections 2023-10-06T04:31:41.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.010+0000 7f807546f700 1 -- 172.21.15.123:0/1456244899 wait complete. 2023-10-06T04:31:41.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.009+0000 7fd9437fe700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7fd9240015f0 0x7fd924003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.010+0000 7fd92affd700 1 -- 172.21.15.123:0/2363718796 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7fd924006b90 con 0x7fd9240015f0 2023-10-06T04:31:41.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.011+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fe5bc002d30 con 0x7fe5b806f830 2023-10-06T04:31:41.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.012+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 shutdown_connections 2023-10-06T04:31:41.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.012+0000 7fa89ea8e700 1 -- 172.21.15.123:0/2574460666 wait complete. 2023-10-06T04:31:41.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.016+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fa6940ae8f0 con 0x7fa67406f7e0 2023-10-06T04:31:41.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7fe5cffff700 1 -- 172.21.15.123:0/3950464250 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fe5bc002d30 con 0x7fe5b806f830 2023-10-06T04:31:41.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7fa6827fc700 1 -- 172.21.15.123:0/1968431105 <== osd.1 v2:172.21.15.123:6816/119720 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fa6940ae8f0 con 0x7fa67406f7e0 2023-10-06T04:31:41.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f6d0804bb80 con 0x7f6cf406f760 2023-10-06T04:31:41.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7fa67406f7e0 msgr2=0x7fa674071bf0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7fa6988bb700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7fa67406f7e0 0x7fa674071bf0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa674066f30 msgr2=0x7fa6740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7fa6988bb700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa674066f30 0x7fa6740693e0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fa6880099b0 tx=0x7fa688016000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.017+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa694156fe0 msgr2=0x7fa694154ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.018+0000 7fa6988bb700 1 --2- 172.21.15.123:0/1968431105 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa694156fe0 0x7fa694154ec0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fa68c00bb60 tx=0x7fa68c009810 comp rx=0 tx=0).stop 2023-10-06T04:31:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.018+0000 7fa692d9d700 1 -- 172.21.15.123:0/1968431105 reap_dead start 2023-10-06T04:31:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.018+0000 7f6ceeffd700 1 -- 172.21.15.123:0/2095273333 <== osd.5 v2:172.21.15.177:6800/119947 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f6d0804bb80 con 0x7f6cf406f760 2023-10-06T04:31:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.018+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 shutdown_connections 2023-10-06T04:31:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.018+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 >> 172.21.15.123:0/1968431105 conn(0x7fa694067660 msgr2=0x7fa6940b0d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.018+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 shutdown_connections 2023-10-06T04:31:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.018+0000 7fa6988bb700 1 -- 172.21.15.123:0/1968431105 wait complete. 2023-10-06T04:31:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.021+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7fe5b806f830 msgr2=0x7fe5b8071c40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.021+0000 7fe5ddacc700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7fe5b806f830 0x7fe5b8071c40 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.022+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f6cf406f760 msgr2=0x7f6cf4071b70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.022+0000 7f6d0d074700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f6cf406f760 0x7f6cf4071b70 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.022+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6cf4066f30 msgr2=0x7f6cf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.022+0000 7f6d0d074700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6cf4066f30 0x7f6cf40693e0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f6cf80030c0 tx=0x7f6cf8017000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.022+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d0806e990 msgr2=0x7f6d0814f9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.022+0000 7f6d0d074700 1 --2- 172.21.15.123:0/2095273333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6d0806e990 0x7f6d0814f9f0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f6cfc00c7e0 tx=0x7f6cfc007e90 comp rx=0 tx=0).stop 2023-10-06T04:31:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.022+0000 7f6d0759e700 1 -- 172.21.15.123:0/2095273333 reap_dead start 2023-10-06T04:31:41.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe5b8066f80 msgr2=0x7fe5b8069430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7fe5ddacc700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe5b8066f80 0x7fe5b8069430 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fe5c0007130 tx=0x7fe5c0007f70 comp rx=0 tx=0).stop 2023-10-06T04:31:41.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d80c43a0 msgr2=0x7fe5d80c9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7fe5ddacc700 1 --2- 172.21.15.123:0/3950464250 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe5d80c43a0 0x7fe5d80c9d70 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe5d0027000 tx=0x7fe5d002a000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7fe5d7fff700 1 -- 172.21.15.123:0/3950464250 reap_dead start 2023-10-06T04:31:41.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 shutdown_connections 2023-10-06T04:31:41.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 >> 172.21.15.123:0/2095273333 conn(0x7f6d080c9d00 msgr2=0x7f6d080cab10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 shutdown_connections 2023-10-06T04:31:41.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 >> 172.21.15.123:0/3950464250 conn(0x7fe5d80680f0 msgr2=0x7fe5d80c8960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 shutdown_connections 2023-10-06T04:31:41.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.023+0000 7f6d0d074700 1 -- 172.21.15.123:0/2095273333 wait complete. 2023-10-06T04:31:41.046 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:41.046 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T04:31:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.030+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 shutdown_connections 2023-10-06T04:31:41.047 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:41.048 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T04:31:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.031+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f40b0002d30 con 0x7f40b406f810 2023-10-06T04:31:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.032+0000 7fe5ddacc700 1 -- 172.21.15.123:0/3950464250 wait complete. 2023-10-06T04:31:41.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.032+0000 7f40c5ffb700 1 -- 172.21.15.123:0/3175826279 <== osd.4 v2:172.21.15.177:6808/119946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f40b0002d30 con 0x7f40b406f810 2023-10-06T04:31:41.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f40b406f810 msgr2=0x7f40b4071c20 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f40b406f810 0x7f40b4071c20 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40b4066f60 msgr2=0x7f40b4069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40b4066f60 0x7f40b4069410 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f40d006f3b0 tx=0x7f40cc017000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d0155b40 msgr2=0x7f40d014a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 --2- 172.21.15.123:0/3175826279 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f40d0155b40 0x7f40d014a390 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f40c000c590 tx=0x7f40c000c8a0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d63a0700 1 -- 172.21.15.123:0/3175826279 reap_dead start 2023-10-06T04:31:41.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 shutdown_connections 2023-10-06T04:31:41.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 >> 172.21.15.123:0/3175826279 conn(0x7f40d00c9d00 msgr2=0x7f40d0159560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.036+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 shutdown_connections 2023-10-06T04:31:41.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.037+0000 7f40d7e03700 1 -- 172.21.15.123:0/3175826279 wait complete. 2023-10-06T04:31:41.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.038+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fd924005c80 con 0x7fd9240015f0 2023-10-06T04:31:41.051 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:41.052 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:41.052 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T04:31:41.053 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T04:31:41.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd92affd700 1 -- 172.21.15.123:0/2363718796 <== osd.6 v2:172.21.15.177:6816/119976 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fd924005c80 con 0x7fd9240015f0 2023-10-06T04:31:41.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7fd9240015f0 msgr2=0x7fd924003aa0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd949a30700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7fd9240015f0 0x7fd924003aa0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd92c0673b0 msgr2=0x7fd92c069860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd949a30700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd92c0673b0 0x7fd92c069860 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd9440c3ee0 tx=0x7fd938007ba0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9440696f0 msgr2=0x7fd9440c1d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd949a30700 1 --2- 172.21.15.123:0/2363718796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd9440696f0 0x7fd9440c1d40 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd94406fae0 tx=0x7fd934017000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.039+0000 7fd9437fe700 1 -- 172.21.15.123:0/2363718796 reap_dead start 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.040+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 shutdown_connections 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.040+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 >> 172.21.15.123:0/2363718796 conn(0x7fd9440680f0 msgr2=0x7fd9440cb3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.040+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 shutdown_connections 2023-10-06T04:31:41.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.040+0000 7fd949a30700 1 -- 172.21.15.123:0/2363718796 wait complete. 2023-10-06T04:31:41.056 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:41.057 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T04:31:41.057 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:41.058 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T04:31:41.058 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2023-10-06T04:31:41.059 INFO:teuthology.orchestra.run.smithi123.stdout:55834574855 2023-10-06T04:31:41.059 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T04:31:41.060 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T04:31:41.242 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.240+0000 7f6da8220700 1 Processor -- start 2023-10-06T04:31:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.241+0000 7f6da8220700 1 -- start start 2023-10-06T04:31:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.242+0000 7f6da8220700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da006dac0 0x7f6da00c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.242+0000 7f6da8220700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6da0155530 0x7f6da00c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.242+0000 7f6da8220700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6da00c6800 con 0x7f6da0155160 2023-10-06T04:31:41.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.242+0000 7f6da8220700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6da0069a00 con 0x7f6da0155530 2023-10-06T04:31:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.242+0000 7f6da8220700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da0069ce0 con 0x7f6da006dac0 2023-10-06T04:31:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.243+0000 7f6da67bd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6da0155530 0x7f6da00c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.243+0000 7f6da5fbc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6da0155160 0x7f6da00691b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56946/0 (socket says 172.21.15.123:56946) 2023-10-06T04:31:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.243+0000 7f6da5fbc700 1 -- 172.21.15.123:0/2741479023 learned_addr learned my addr 172.21.15.123:0/2741479023 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.243+0000 7f6da57bb700 1 --2- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da006dac0 0x7f6da00c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.244+0000 7f6da67bd700 1 -- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da006dac0 msgr2=0x7f6da00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.244+0000 7f6da67bd700 1 --2- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da006dac0 0x7f6da00c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.244+0000 7f6da67bd700 1 -- 172.21.15.123:0/2741479023 >> v1:172.21.15.123:6789/0 conn(0x7f6da0155160 legacy=0x7f6da00691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.244+0000 7f6da67bd700 1 -- 172.21.15.123:0/2741479023 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6da0069fc0 con 0x7f6da0155530 2023-10-06T04:31:41.246 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.245+0000 7f6da67bd700 1 --2- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da0155530 0x7f6da00c8c00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009e60 tx=0x7f6d98009780 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9c55a0f502514ca server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.245+0000 7f6da4fba700 1 -- 172.21.15.123:0/2741479023 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800e070 con 0x7f6da0155530 2023-10-06T04:31:41.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.246+0000 7f6da4fba700 1 -- 172.21.15.123:0/2741479023 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d9800ba60 con 0x7f6da0155530 2023-10-06T04:31:41.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.246+0000 7f6da4fba700 1 -- 172.21.15.123:0/2741479023 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d9800be00 con 0x7f6da0155530 2023-10-06T04:31:41.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 -- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da0155530 msgr2=0x7f6da00c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.248 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 --2- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da0155530 0x7f6da00c8c00 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f6d98009e60 tx=0x7f6d98009780 comp rx=0 tx=0).stop 2023-10-06T04:31:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 -- 172.21.15.123:0/2741479023 shutdown_connections 2023-10-06T04:31:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 --2- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da0155530 0x7f6da00c8c00 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 --2- 172.21.15.123:0/2741479023 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da006dac0 0x7f6da00c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 -- 172.21.15.123:0/2741479023 >> 172.21.15.123:0/2741479023 conn(0x7f6da0067660 msgr2=0x7f6da0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 -- 172.21.15.123:0/2741479023 shutdown_connections 2023-10-06T04:31:41.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.247+0000 7f6da8220700 1 -- 172.21.15.123:0/2741479023 wait complete. 2023-10-06T04:31:41.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.249+0000 7f6da8220700 1 Processor -- start 2023-10-06T04:31:41.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.250+0000 7f6da8220700 1 -- start start 2023-10-06T04:31:41.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da8220700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da006dac0 0x7f6da0148c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da8220700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da0155160 0x7f6da016d500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da8220700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6da0155530 0x7f6da00c1d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da8220700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da016da40 con 0x7f6da0155530 2023-10-06T04:31:41.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da8220700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00af0f0 con 0x7f6da006dac0 2023-10-06T04:31:41.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da8220700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6da00c2260 con 0x7f6da0155160 2023-10-06T04:31:41.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da67bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6da0155530 0x7f6da00c1d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da67bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6da0155530 0x7f6da00c1d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43224/0 (socket says 172.21.15.123:43224) 2023-10-06T04:31:41.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.251+0000 7f6da67bd700 1 -- 172.21.15.123:0/377103829 learned_addr learned my addr 172.21.15.123:0/377103829 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.252+0000 7f6da67bd700 1 -- 172.21.15.123:0/377103829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da0155160 msgr2=0x7f6da016d500 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:31:41.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.252+0000 7f6da67bd700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da0155160 0x7f6da016d500 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.252+0000 7f6da67bd700 1 -- 172.21.15.123:0/377103829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da006dac0 msgr2=0x7f6da0148c70 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:31:41.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.252+0000 7f6da67bd700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da006dac0 0x7f6da0148c70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.252+0000 7f6da67bd700 1 -- 172.21.15.123:0/377103829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d98009a20 con 0x7f6da0155530 2023-10-06T04:31:41.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.253+0000 7f6da67bd700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6da0155530 0x7f6da00c1d20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6d980129a0 tx=0x7f6d98018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.268+0000 7f6d95ffb700 1 -- 172.21.15.123:0/377103829 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d980070c0 con 0x7f6da0155530 2023-10-06T04:31:41.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.268+0000 7f6d95ffb700 1 -- 172.21.15.123:0/377103829 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d98026460 con 0x7f6da0155530 2023-10-06T04:31:41.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.268+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6da0149250 con 0x7f6da0155530 2023-10-06T04:31:41.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.268+0000 7f6d95ffb700 1 -- 172.21.15.123:0/377103829 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d98026800 con 0x7f6da0155530 2023-10-06T04:31:41.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.269+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6da0149810 con 0x7f6da0155530 2023-10-06T04:31:41.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.270+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6da006c3d0 con 0x7f6da0155530 2023-10-06T04:31:41.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.271+0000 7f6d95ffb700 1 -- 172.21.15.123:0/377103829 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6d9801d420 con 0x7f6da0155530 2023-10-06T04:31:41.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.272+0000 7f6d95ffb700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6d8c0671a0 0x7f6d8c069650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.272+0000 7f6d95ffb700 1 -- 172.21.15.123:0/377103829 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6d98087400 con 0x7f6da0155530 2023-10-06T04:31:41.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.276+0000 7f6da5fbc700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6d8c0671a0 0x7f6d8c069650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.276+0000 7f6da5fbc700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6d8c0671a0 0x7f6d8c069650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c007fd0 tx=0x7f6d9c007a50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.282+0000 7f6d95ffb700 1 -- 172.21.15.123:0/377103829 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6d98054570 con 0x7f6da0155530 2023-10-06T04:31:41.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.291+0000 7f4edfc40700 1 Processor -- start 2023-10-06T04:31:41.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.291+0000 7f4edfc40700 1 -- start start 2023-10-06T04:31:41.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.293+0000 7f4edfc40700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4ed80bf330 0x7f4ed80691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.293+0000 7f4edfc40700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed80c2f00 con 0x7f4ed80696f0 2023-10-06T04:31:41.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.293+0000 7f4edfc40700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ed8069ae0 con 0x7f4ed80cca10 2023-10-06T04:31:41.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.293+0000 7f4edfc40700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ed80c7cf0 con 0x7f4ed80bf330 2023-10-06T04:31:41.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.293+0000 7f4ede1dd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4ed80696f0 0x7f4ed80c74a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56948/0 (socket says 172.21.15.123:56948) 2023-10-06T04:31:41.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.293+0000 7f4ede1dd700 1 -- 172.21.15.123:0/1633705659 learned_addr learned my addr 172.21.15.123:0/1633705659 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:41.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f0208761700 1 Processor -- start 2023-10-06T04:31:41.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edd9dc700 1 --2- 172.21.15.123:0/1633705659 >> v2:172.21.15.177:3300/0 conn(0x7f4ed80bf330 0x7f4ed80691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2829435011 0 0) 0x7f4ed80c2f00 con 0x7f4ed80696f0 2023-10-06T04:31:41.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ec4003570 con 0x7f4ed80696f0 2023-10-06T04:31:41.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2716276539 0 0) 0x7f4ed8069ae0 con 0x7f4ed80cca10 2023-10-06T04:31:41.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ed80c2f00 con 0x7f4ed80cca10 2023-10-06T04:31:41.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1820361620 0 0) 0x7f4ec4003570 con 0x7f4ed80696f0 2023-10-06T04:31:41.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 >> v2:172.21.15.177:3300/0 conn(0x7f4ed80bf330 msgr2=0x7f4ed80691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 --2- 172.21.15.123:0/1633705659 >> v2:172.21.15.177:3300/0 conn(0x7f4ed80bf330 0x7f4ed80691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 >> v1:172.21.15.177:6789/0 conn(0x7f4ed80cca10 legacy=0x7f4ed80c3390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed80c7fd0 con 0x7f4ed80696f0 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f4ed00017f0 con 0x7f4ed80696f0 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 >> v1:172.21.15.123:6789/0 conn(0x7f4ed80696f0 legacy=0x7f4ed80c74a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4003c40 0x7f4ec4004030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec4004810 0x7f4ec4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec40052e0 0x7f4ec400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed80c7cf0 con 0x7f4ec4003c40 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400c1b0 con 0x7f4ec40052e0 2023-10-06T04:31:41.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.294+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ec400c370 con 0x7f4ec4004810 2023-10-06T04:31:41.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4edd1db700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec4004810 0x7f4ec4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4ede1dd700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec40052e0 0x7f4ec400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4edd1db700 1 -- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec40052e0 msgr2=0x7f4ec400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4edd1db700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ec40052e0 0x7f4ec400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4edd1db700 1 -- 172.21.15.123:0/1633705659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4003c40 msgr2=0x7f4ec4004030 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4edd1db700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4003c40 0x7f4ec4004030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4edd1db700 1 -- 172.21.15.123:0/1633705659 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ec4003570 con 0x7f4ec4004810 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4ede1dd700 1 -- 172.21.15.123:0/1633705659 reap_dead start 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.295+0000 7f4edd1db700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec4004810 0x7f4ec4004c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001a40 tx=0x7f4ed4007d10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bcacb6b76b6f8644 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.296+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400c5d0 con 0x7f4ec4004810 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.296+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ed400c730 con 0x7f4ec4004810 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.296+0000 7f4edc9da700 1 -- 172.21.15.123:0/1633705659 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ed400caf0 con 0x7f4ec4004810 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.297+0000 7f4edfc40700 1 -- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec4004810 msgr2=0x7f4ec4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.297+0000 7f4edfc40700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec4004810 0x7f4ec4004c80 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001a40 tx=0x7f4ed4007d10 comp rx=0 tx=0).stop 2023-10-06T04:31:41.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.296+0000 7f0208761700 1 -- start start 2023-10-06T04:31:41.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.298+0000 7f0208761700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f02001555f0 0x7f02000c7220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.298+0000 7f0208761700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02001559c0 0x7f02000695d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.298+0000 7f0208761700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f020006f900 con 0x7f0200064500 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.298+0000 7f0208761700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f020006e5c0 con 0x7f02001555f0 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.298+0000 7f0208761700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000ca690 con 0x7f02001559c0 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.298+0000 7f0206cfe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0200064500 0x7f020006deb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:53318/0 (socket says 172.21.15.123:53318) 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.298+0000 7f0206cfe700 1 -- 172.21.15.123:0/1197387778 learned_addr learned my addr 172.21.15.123:0/1197387778 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163442574 0 0) 0x7f020006f900 con 0x7f0200064500 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01ec003570 con 0x7f0200064500 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f4edfc40700 1 -- 172.21.15.123:0/1633705659 shutdown_connections 2023-10-06T04:31:41.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f4edfc40700 1 --2- 172.21.15.123:0/1633705659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ec4004810 0x7f4ec4004c80 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f4edfc40700 1 -- 172.21.15.123:0/1633705659 >> 172.21.15.123:0/1633705659 conn(0x7f4ed80680f0 msgr2=0x7f4ed80cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f0205cfc700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02001559c0 0x7f02000695d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02064fd700 1 --2- 172.21.15.123:0/1197387778 >> v2:172.21.15.177:3300/0 conn(0x7f02001555f0 0x7f02000c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3613150639 0 0) 0x7f01ec003570 con 0x7f0200064500 2023-10-06T04:31:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02001559c0 msgr2=0x7f02000695d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f02001559c0 0x7f02000695d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 >> v2:172.21.15.177:3300/0 conn(0x7f02001555f0 msgr2=0x7f02000c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 --2- 172.21.15.123:0/1197387778 >> v2:172.21.15.177:3300/0 conn(0x7f02001555f0 0x7f02000c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 --> v1:172.21.15.177:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f020006f620 con 0x7f0200064500 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 <== mon.1 v1:172.21.15.177:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f01f80017f0 con 0x7f0200064500 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 >> v1:172.21.15.177:6789/0 conn(0x7f0200064500 legacy=0x7f020006deb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ec003ae0 0x7f01ec003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec004810 0x7f01ec004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ec0051c0 0x7f01ec00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000ca690 con 0x7f01ec004810 2023-10-06T04:31:41.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec00c0b0 con 0x7f01ec0051c0 2023-10-06T04:31:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01ec00c270 con 0x7f01ec003ae0 2023-10-06T04:31:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.299+0000 7f0206cfe700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ec0051c0 0x7f01ec00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f02064fd700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ec003ae0 0x7f01ec003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f0206cfe700 1 -- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ec003ae0 msgr2=0x7f01ec003ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f0206cfe700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01ec003ae0 0x7f01ec003ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f0206cfe700 1 -- 172.21.15.123:0/1197387778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec004810 msgr2=0x7f01ec004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f0206cfe700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec004810 0x7f01ec004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f0206cfe700 1 -- 172.21.15.123:0/1197387778 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01ec003570 con 0x7f01ec0051c0 2023-10-06T04:31:41.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f4edfc40700 1 -- 172.21.15.123:0/1633705659 shutdown_connections 2023-10-06T04:31:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f0206cfe700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ec0051c0 0x7f01ec00ba60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01f800a870 tx=0x7f01f8002140 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5f0c8c356561500f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f80070f0 con 0x7f01ec0051c0 2023-10-06T04:31:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01f8007250 con 0x7f01ec0051c0 2023-10-06T04:31:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f02054fb700 1 -- 172.21.15.123:0/1197387778 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01f8007610 con 0x7f01ec0051c0 2023-10-06T04:31:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.301+0000 7f0208761700 1 -- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ec0051c0 msgr2=0x7f01ec00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.301+0000 7f0208761700 1 --2- 172.21.15.123:0/1197387778 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01ec0051c0 0x7f01ec00ba60 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f01f800a870 tx=0x7f01f8002140 comp rx=0 tx=0).stop 2023-10-06T04:31:41.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.301+0000 7f0206cfe700 1 -- 172.21.15.123:0/1197387778 reap_dead start 2023-10-06T04:31:41.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.301+0000 7f0208761700 1 -- 172.21.15.123:0/1197387778 shutdown_connections 2023-10-06T04:31:41.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.301+0000 7f0208761700 1 -- 172.21.15.123:0/1197387778 >> 172.21.15.123:0/1197387778 conn(0x7f0200068130 msgr2=0x7f02000cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.300+0000 7f4edfc40700 1 -- 172.21.15.123:0/1633705659 wait complete. 2023-10-06T04:31:41.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.302+0000 7f4edfc40700 1 Processor -- start 2023-10-06T04:31:41.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.302+0000 7f0208761700 1 -- 172.21.15.123:0/1197387778 shutdown_connections 2023-10-06T04:31:41.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.302+0000 7f0208761700 1 -- 172.21.15.123:0/1197387778 wait complete. 2023-10-06T04:31:41.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.302+0000 7f2dc8caf700 1 Processor -- start 2023-10-06T04:31:41.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.303+0000 7f2dc8caf700 1 -- start start 2023-10-06T04:31:41.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f2dc8caf700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2dc40c9f10 0x7f2dc40cadd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f2dc8caf700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc40cbd60 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f2dc8caf700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2dc40646c0 con 0x7f2dc415da00 2023-10-06T04:31:41.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f2dc8caf700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2dc40cb4c0 con 0x7f2dc40c9f10 2023-10-06T04:31:41.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.303+0000 7f0208761700 1 Processor -- start 2023-10-06T04:31:41.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.303+0000 7f0208761700 1 -- start start 2023-10-06T04:31:41.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f0208761700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec0051c0 0x7f02001616a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f0208761700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200151ca0 0x7f02000afbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f0208761700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02000b0540 0x7f02000c5460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f0208761700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c4fd0 con 0x7f01ec0051c0 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f0208761700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02000b00f0 con 0x7f02000b0540 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f0208761700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02000c43c0 con 0x7f0200151ca0 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.303+0000 7f4edfc40700 1 -- start start 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f4edfc40700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4004810 0x7f4ed80c6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f4edfc40700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ed8177850 0x7f4ed80b1280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f4edfc40700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ed8177cc0 0x7f4ed80b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f4edfc40700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8176a10 con 0x7f4ec4004810 2023-10-06T04:31:41.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f4edfc40700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8176cd0 con 0x7f4ed8177cc0 2023-10-06T04:31:41.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.304+0000 7f4edfc40700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ed8176fb0 con 0x7f4ed8177850 2023-10-06T04:31:41.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f2dc2d9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2dc40c9f10 0x7f2dc40cadd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f2dc2d9d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2dc40c9f10 0x7f2dc40cadd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41130/0 (socket says 172.21.15.123:41130) 2023-10-06T04:31:41.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f2dc2d9d700 1 -- 172.21.15.123:0/22236087 learned_addr learned my addr 172.21.15.123:0/22236087 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0206cfe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02000b0540 0x7f02000c5460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f4edd1db700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ed8177850 0x7f4ed80b1280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0206cfe700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02000b0540 0x7f02000c5460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41138/0 (socket says 172.21.15.123:41138) 2023-10-06T04:31:41.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0206cfe700 1 -- 172.21.15.123:0/4268418006 learned_addr learned my addr 172.21.15.123:0/4268418006 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.314 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f4ede1dd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ed8177cc0 0x7f4ed80b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f4edd1db700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ed8177850 0x7f4ed80b1280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46482/0 (socket says 172.21.15.123:46482) 2023-10-06T04:31:41.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f4edd1db700 1 -- 172.21.15.123:0/3379555403 learned_addr learned my addr 172.21.15.123:0/3379555403 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0205cfc700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200151ca0 0x7f02000afbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f2dc2d9d700 1 -- 172.21.15.123:0/22236087 >> v1:172.21.15.177:6789/0 conn(0x7f2dc415da00 legacy=0x7f2dc406dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.315 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f2dc2d9d700 1 -- 172.21.15.123:0/22236087 >> v1:172.21.15.123:6789/0 conn(0x7f2dc40c6bd0 legacy=0x7f2dc415d2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f2dc2d9d700 1 -- 172.21.15.123:0/22236087 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2dc40cb7a0 con 0x7f2dc40c9f10 2023-10-06T04:31:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f02064fd700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec0051c0 0x7f02001616a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0205cfc700 1 -- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02000b0540 msgr2=0x7f02000c5460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0205cfc700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02000b0540 0x7f02000c5460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0205cfc700 1 -- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec0051c0 msgr2=0x7f02001616a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0205cfc700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec0051c0 0x7f02001616a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f0205cfc700 1 -- 172.21.15.123:0/4268418006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02000c46a0 con 0x7f0200151ca0 2023-10-06T04:31:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.305+0000 7f02064fd700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec0051c0 0x7f02001616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f0205cfc700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200151ca0 0x7f02000afbb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0200071110 tx=0x7f01fc00b320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f4edd9dc700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4004810 0x7f4ed80c6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f01f67fc700 1 -- 172.21.15.123:0/4268418006 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc002820 con 0x7f0200151ca0 2023-10-06T04:31:41.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f01f67fc700 1 -- 172.21.15.123:0/4268418006 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01fc002980 con 0x7f0200151ca0 2023-10-06T04:31:41.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f01f67fc700 1 -- 172.21.15.123:0/4268418006 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01fc002d80 con 0x7f0200151ca0 2023-10-06T04:31:41.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02000c4980 con 0x7f0200151ca0 2023-10-06T04:31:41.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02000b09e0 con 0x7f0200151ca0 2023-10-06T04:31:41.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f4edd9dc700 1 -- 172.21.15.123:0/3379555403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ed8177850 msgr2=0x7f4ed80b1280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f4edd9dc700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ed8177850 0x7f4ed80b1280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f4edd9dc700 1 -- 172.21.15.123:0/3379555403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ed8177cc0 msgr2=0x7f4ed80b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f4edd9dc700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ed8177cc0 0x7f4ed80b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f4edd9dc700 1 -- 172.21.15.123:0/3379555403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ed8177290 con 0x7f4ec4004810 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.306+0000 7f4edd9dc700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4004810 0x7f4ed80c6950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ed804d2d0 tx=0x7f4ec8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.307+0000 7f4ecdffb700 1 -- 172.21.15.123:0/3379555403 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec80125f0 con 0x7f4ec4004810 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.307+0000 7f4ecdffb700 1 -- 172.21.15.123:0/3379555403 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ec8012750 con 0x7f4ec4004810 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.307+0000 7f4ecdffb700 1 -- 172.21.15.123:0/3379555403 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8012af0 con 0x7f4ec4004810 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.307+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ed8177570 con 0x7f4ec4004810 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.307+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ed80c7050 con 0x7f4ec4004810 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.308+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ed806c0f0 con 0x7f4ec4004810 2023-10-06T04:31:41.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.308+0000 7f2dc2d9d700 1 --2- 172.21.15.123:0/22236087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc40c9f10 0x7f2dc40cadd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2db40099b0 tx=0x7f2db4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e6693a14732b205 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.309+0000 7f01f67fc700 1 -- 172.21.15.123:0/4268418006 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01fc01f3f0 con 0x7f0200151ca0 2023-10-06T04:31:41.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.310+0000 7f2dc159a700 1 -- 172.21.15.123:0/22236087 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db400b7e0 con 0x7f2dc40c9f10 2023-10-06T04:31:41.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.310+0000 7f2dc159a700 1 -- 172.21.15.123:0/22236087 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db400b940 con 0x7f2dc40c9f10 2023-10-06T04:31:41.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.310+0000 7f2dc159a700 1 -- 172.21.15.123:0/22236087 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db400bce0 con 0x7f2dc40c9f10 2023-10-06T04:31:41.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.311+0000 7fbcc83d7700 1 Processor -- start 2023-10-06T04:31:41.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.311+0000 7f4ecdffb700 1 -- 172.21.15.123:0/3379555403 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4ec8013480 con 0x7f4ec4004810 2023-10-06T04:31:41.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.311+0000 7f2dc8caf700 1 -- 172.21.15.123:0/22236087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc40c9f10 msgr2=0x7f2dc40cadd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.311+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/22236087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc40c9f10 0x7f2dc40cadd0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2db40099b0 tx=0x7f2db4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:41.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.311+0000 7f4ecdffb700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4ebc067000 0x7f4ebc0694b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.311+0000 7fbcc83d7700 1 -- start start 2023-10-06T04:31:41.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc83d7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbcc0155160 0x7fbcc00691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc83d7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbcc00c8dd0 0x7fbcc00c6c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc83d7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcc00c6800 con 0x7fbcc006dac0 2023-10-06T04:31:41.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc83d7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc0069a00 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc83d7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcc0069ce0 con 0x7fbcc0155160 2023-10-06T04:31:41.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc5972700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbcc006dac0 0x7fbcc00c86c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56970/0 (socket says 172.21.15.123:56970) 2023-10-06T04:31:41.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc5972700 1 -- 172.21.15.123:0/1984770409 learned_addr learned my addr 172.21.15.123:0/1984770409 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:41.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.312+0000 7fbcc6173700 1 --2- 172.21.15.123:0/1984770409 >> v2:172.21.15.177:3300/0 conn(0x7fbcc0155160 0x7fbcc00691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.309+0000 7f01f67fc700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01e4067080 0x7f01e4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.309+0000 7f01f67fc700 1 -- 172.21.15.123:0/4268418006 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f01fc0844f0 con 0x7f0200151ca0 2023-10-06T04:31:41.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.310+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02000c4e20 con 0x7f0200151ca0 2023-10-06T04:31:41.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7fbcc6974700 1 --2- 172.21.15.123:0/1984770409 >> v2:172.21.15.123:3300/0 conn(0x7fbcc00c8dd0 0x7fbcc00c6c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7fbcc6974700 1 -- 172.21.15.123:0/1984770409 >> v2:172.21.15.177:3300/0 conn(0x7fbcc0155160 msgr2=0x7fbcc00691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7fbcc6974700 1 --2- 172.21.15.123:0/1984770409 >> v2:172.21.15.177:3300/0 conn(0x7fbcc0155160 0x7fbcc00691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7fbcc6974700 1 -- 172.21.15.123:0/1984770409 >> v1:172.21.15.123:6789/0 conn(0x7fbcc006dac0 legacy=0x7fbcc00c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7fbcc6974700 1 -- 172.21.15.123:0/1984770409 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcc0069fc0 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7f4ecdffb700 1 -- 172.21.15.123:0/3379555403 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4ec8055300 con 0x7f4ec4004810 2023-10-06T04:31:41.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7f4edd1db700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4ebc067000 0x7f4ebc0694b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.314+0000 7fbcc6974700 1 --2- 172.21.15.123:0/1984770409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc00c6c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbcc006a440 tx=0x7fbcb80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8cbd22897793a22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.314+0000 7f4edd1db700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4ebc067000 0x7f4ebc0694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001500 tx=0x7f4ed4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7f02064fd700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01e4067080 0x7f01e4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.314+0000 7f02064fd700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01e4067080 0x7f01e4069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f0006fd0 tx=0x7f01f000b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7f2dc8caf700 1 -- 172.21.15.123:0/22236087 shutdown_connections 2023-10-06T04:31:41.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/22236087 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc40c9f10 0x7f2dc40cadd0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.313+0000 7f2dc8caf700 1 -- 172.21.15.123:0/22236087 >> 172.21.15.123:0/22236087 conn(0x7f2dc4068130 msgr2=0x7f2dc40cc340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.314+0000 7f2dc8caf700 1 -- 172.21.15.123:0/22236087 shutdown_connections 2023-10-06T04:31:41.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.315+0000 7f2dc8caf700 1 -- 172.21.15.123:0/22236087 wait complete. 2023-10-06T04:31:41.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.315+0000 7f12c9c0f700 1 Processor -- start 2023-10-06T04:31:41.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.315+0000 7f12c9c0f700 1 -- start start 2023-10-06T04:31:41.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c9c0f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f12c406dac0 0x7f12c40c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c9c0f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12c4155530 0x7f12c40c8c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c9c0f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12c40c6800 con 0x7f12c4155160 2023-10-06T04:31:41.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c9c0f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c4069a00 con 0x7f12c4155530 2023-10-06T04:31:41.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c9c0f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12c4069ce0 con 0x7f12c406dac0 2023-10-06T04:31:41.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7fbcc5171700 1 -- 172.21.15.123:0/1984770409 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8011630 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c3fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f12c4155530 0x7f12c40c8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7fbcc5171700 1 -- 172.21.15.123:0/1984770409 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb8011790 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7fbcc5171700 1 -- 172.21.15.123:0/1984770409 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb8011b50 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c37fe700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f12c4155160 0x7f12c40691b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:53330/0 (socket says 172.21.15.123:53330) 2023-10-06T04:31:41.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c2ffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f12c406dac0 0x7f12c40c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c37fe700 1 -- 172.21.15.123:0/430981369 learned_addr learned my addr 172.21.15.123:0/430981369 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:41.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c27fc700 1 -- 172.21.15.123:0/430981369 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839077747 0 0) 0x7f12c40c6800 con 0x7f12c4155160 2023-10-06T04:31:41.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c27fc700 1 -- 172.21.15.123:0/430981369 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f12ac003570 con 0x7f12c4155160 2023-10-06T04:31:41.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f2dc8caf700 1 Processor -- start 2023-10-06T04:31:41.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c3fff700 1 -- 172.21.15.123:0/430981369 >> v2:172.21.15.177:3300/0 conn(0x7f12c406dac0 msgr2=0x7f12c40c86c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c3fff700 1 --2- 172.21.15.123:0/430981369 >> v2:172.21.15.177:3300/0 conn(0x7f12c406dac0 0x7f12c40c86c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c3fff700 1 -- 172.21.15.123:0/430981369 >> v1:172.21.15.177:6789/0 conn(0x7f12c4155160 legacy=0x7f12c40691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c3fff700 1 -- 172.21.15.123:0/430981369 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12c4069fc0 con 0x7f12c4155530 2023-10-06T04:31:41.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f12c3fff700 1 --2- 172.21.15.123:0/430981369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c4155530 0x7f12c40c8c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009bd0 tx=0x7f12bc009c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef4121e6fd30c953 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7fbcc83d7700 1 -- 172.21.15.123:0/1984770409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 msgr2=0x7fbcc00c6c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/1984770409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc00c6c90 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fbcc006a440 tx=0x7fbcb80091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7f12c27fc700 1 -- 172.21.15.123:0/430981369 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc00d9f0 con 0x7f12c4155530 2023-10-06T04:31:41.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f12c27fc700 1 -- 172.21.15.123:0/430981369 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12bc00db50 con 0x7f12c4155530 2023-10-06T04:31:41.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f12c27fc700 1 -- 172.21.15.123:0/430981369 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12bc010430 con 0x7f12c4155530 2023-10-06T04:31:41.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.316+0000 7f2dc8caf700 1 -- start start 2023-10-06T04:31:41.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7f2dc8caf700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2dc40c6bd0 0x7f2dc40c4a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7f2dc8caf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dc40c9f10 0x7f2dc4179fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7f2dc8caf700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc415da00 0x7f2dc417a4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7f2dc8caf700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc40c4580 con 0x7f2dc40c9f10 2023-10-06T04:31:41.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7f2dc8caf700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4162870 con 0x7f2dc415da00 2023-10-06T04:31:41.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.317+0000 7f2dc8caf700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2dc4162590 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7fbcc83d7700 1 -- 172.21.15.123:0/1984770409 shutdown_connections 2023-10-06T04:31:41.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/1984770409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc00c6c90 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/1984770409 >> v2:172.21.15.177:3300/0 conn(0x7fbcc0155160 0x7fbcc00691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7fbcc83d7700 1 -- 172.21.15.123:0/1984770409 >> 172.21.15.123:0/1984770409 conn(0x7fbcc0067660 msgr2=0x7fbcc0067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f4ecdffb700 1 -- 172.21.15.123:0/3379555403 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4ec8089060 con 0x7f4ec4004810 2023-10-06T04:31:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7fbcc83d7700 1 -- 172.21.15.123:0/1984770409 shutdown_connections 2023-10-06T04:31:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2dc40c6bd0 0x7f2dc40c4a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f2dc259c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2dc40c6bd0 0x7f2dc40c4a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46496/0 (socket says 172.21.15.123:46496) 2023-10-06T04:31:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f2dc259c700 1 -- 172.21.15.123:0/1622693149 learned_addr learned my addr 172.21.15.123:0/1622693149 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f12c9c0f700 1 -- 172.21.15.123:0/430981369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c4155530 msgr2=0x7f12c40c8c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/430981369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c4155530 0x7f12c40c8c00 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009bd0 tx=0x7f12bc009c00 comp rx=0 tx=0).stop 2023-10-06T04:31:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7f2dc1d9b700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dc40c9f10 0x7f2dc4179fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2dc259c700 1 -- 172.21.15.123:0/1622693149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc415da00 msgr2=0x7f2dc417a4e0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2dc259c700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc415da00 0x7f2dc417a4e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2dc259c700 1 -- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dc40c9f10 msgr2=0x7f2dc4179fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2dc259c700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dc40c9f10 0x7f2dc4179fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2dc259c700 1 -- 172.21.15.123:0/1622693149 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2db40094f0 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2dc259c700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2dc40c6bd0 0x7f2dc40c4a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b630 tx=0x7f2db80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2db27fc700 1 -- 172.21.15.123:0/1622693149 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800f510 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2db27fc700 1 -- 172.21.15.123:0/1622693149 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2db800f690 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2db27fc700 1 -- 172.21.15.123:0/1622693149 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2db800fa50 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2dc40c3f30 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.318+0000 7fbcc83d7700 1 -- 172.21.15.123:0/1984770409 wait complete. 2023-10-06T04:31:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7fbcc83d7700 1 Processor -- start 2023-10-06T04:31:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f12c9c0f700 1 -- 172.21.15.123:0/430981369 shutdown_connections 2023-10-06T04:31:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/430981369 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c4155530 0x7f12c40c8c00 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.339 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/430981369 >> v2:172.21.15.177:3300/0 conn(0x7f12c406dac0 0x7f12c40c86c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7f12c9c0f700 1 -- 172.21.15.123:0/430981369 >> 172.21.15.123:0/430981369 conn(0x7f12c4067660 msgr2=0x7f12c4067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7f01f67fc700 1 -- 172.21.15.123:0/4268418006 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f01fc0550f0 con 0x7f0200151ca0 2023-10-06T04:31:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.319+0000 7fbcc83d7700 1 -- start start 2023-10-06T04:31:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc83d7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc006dac0 0x7fbcc01701b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc83d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc0177430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc83d7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc0155160 0x7fbcc0179860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc83d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00c3e30 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc83d7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc00c3260 con 0x7fbcc0155160 2023-10-06T04:31:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc83d7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcc01706f0 con 0x7fbcc006dac0 2023-10-06T04:31:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.321+0000 7f12c9c0f700 1 -- 172.21.15.123:0/430981369 shutdown_connections 2023-10-06T04:31:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc5972700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc0177430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc5972700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc0177430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43316/0 (socket says 172.21.15.123:43316) 2023-10-06T04:31:41.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.320+0000 7fbcc5972700 1 -- 172.21.15.123:0/4003773312 learned_addr learned my addr 172.21.15.123:0/4003773312 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.321+0000 7f0df9d24700 1 Processor -- start 2023-10-06T04:31:41.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.321+0000 7f12c9c0f700 1 -- 172.21.15.123:0/430981369 wait complete. 2023-10-06T04:31:41.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f12c9c0f700 1 Processor -- start 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7fbcc5972700 1 -- 172.21.15.123:0/4003773312 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc006dac0 msgr2=0x7fbcc01701b0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7fbcc5972700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc006dac0 0x7fbcc01701b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7fbcc5972700 1 -- 172.21.15.123:0/4003773312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc0155160 msgr2=0x7fbcc0179860 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7fbcc5972700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc0155160 0x7fbcc0179860 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7fbcc5972700 1 -- 172.21.15.123:0/4003773312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcb8009420 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f12c9c0f700 1 -- start start 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c9c0f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c406dac0 0x7f12c40a5770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c9c0f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12c4155160 0x7f12c40c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c9c0f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12c4155530 0x7f12c4175da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c9c0f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c4176db0 con 0x7f12c406dac0 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c9c0f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12c41762e0 con 0x7f12c4155160 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c9c0f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12c40c3130 con 0x7f12c4155530 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c406dac0 0x7f12c40a5770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c406dac0 0x7f12c40a5770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43322/0 (socket says 172.21.15.123:43322) 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c37fe700 1 -- 172.21.15.123:0/3494489094 learned_addr learned my addr 172.21.15.123:0/3494489094 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.321+0000 7f0df9d24700 1 -- start start 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df9d24700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0df41560e0 0x7f0df406b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df9d24700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df4068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df9d24700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0df406ab80 con 0x7f0df4068770 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df9d24700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0df406f1f0 con 0x7f0df41560e0 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df9d24700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4069150 con 0x7f0df406e960 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df3fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df4068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df3fff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df4068c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46512/0 (socket says 172.21.15.123:46512) 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7f0df3fff700 1 -- 172.21.15.123:0/26612400 learned_addr learned my addr 172.21.15.123:0/26612400 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7f12c3fff700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12c4155530 0x7f12c4175da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f12c37fe700 1 -- 172.21.15.123:0/3494489094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12c4155530 msgr2=0x7f12c4175da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f12c37fe700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12c4155530 0x7f12c4175da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f12c37fe700 1 -- 172.21.15.123:0/3494489094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12c4155160 msgr2=0x7f12c40c5de0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f12c37fe700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12c4155160 0x7f12c40c5de0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f12c37fe700 1 -- 172.21.15.123:0/3494489094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12bc009790 con 0x7f12c406dac0 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7fbcc5972700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc0177430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400d990 tx=0x7fbcb4006090 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.322+0000 7fbcbe7fc700 1 -- 172.21.15.123:0/4003773312 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb400de00 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7fbcbe7fc700 1 -- 172.21.15.123:0/4003773312 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcb4008950 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.323+0000 7fbcbe7fc700 1 -- 172.21.15.123:0/4003773312 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcb4008d10 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df3fff700 1 -- 172.21.15.123:0/26612400 >> v2:172.21.15.177:3300/0 conn(0x7f0df41560e0 msgr2=0x7f0df406b010 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:31:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df3fff700 1 --2- 172.21.15.123:0/26612400 >> v2:172.21.15.177:3300/0 conn(0x7f0df41560e0 0x7f0df406b010 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df3fff700 1 -- 172.21.15.123:0/26612400 >> v1:172.21.15.177:6789/0 conn(0x7f0df4068770 legacy=0x7f0df406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df3fff700 1 -- 172.21.15.123:0/26612400 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df406ef10 con 0x7f0df406e960 2023-10-06T04:31:41.347 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f12c37fe700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c406dac0 0x7f12c40a5770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f12b400c7f0 tx=0x7f12b4007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df3fff700 1 --2- 172.21.15.123:0/26612400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df4068c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00d1a0 tx=0x7f0dec00ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0682faece881e52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df2ffd700 1 -- 172.21.15.123:0/26612400 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec010070 con 0x7f0df406e960 2023-10-06T04:31:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcc00c3500 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcc00c3ac0 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df2ffd700 1 -- 172.21.15.123:0/26612400 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dec00de80 con 0x7f0df406e960 2023-10-06T04:31:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.324+0000 7f0df2ffd700 1 -- 172.21.15.123:0/26612400 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dec00b320 con 0x7f0df406e960 2023-10-06T04:31:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.325+0000 7f0df9d24700 1 -- 172.21.15.123:0/26612400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 msgr2=0x7f0df4068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.325+0000 7f0df9d24700 1 --2- 172.21.15.123:0/26612400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df4068c10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0dec00d1a0 tx=0x7f0dec00ba50 comp rx=0 tx=0).stop 2023-10-06T04:31:41.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.325+0000 7f0df9d24700 1 -- 172.21.15.123:0/26612400 shutdown_connections 2023-10-06T04:31:41.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.325+0000 7f0df9d24700 1 --2- 172.21.15.123:0/26612400 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df4068c10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.325+0000 7f0df9d24700 1 --2- 172.21.15.123:0/26612400 >> v2:172.21.15.177:3300/0 conn(0x7f0df41560e0 0x7f0df406b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.325+0000 7f0df9d24700 1 -- 172.21.15.123:0/26612400 >> 172.21.15.123:0/26612400 conn(0x7f0df40cd010 msgr2=0x7f0df40cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.326+0000 7fbcbe7fc700 1 -- 172.21.15.123:0/4003773312 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbcb4010670 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.327+0000 7fbcbe7fc700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbcac066f30 0x7fbcac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.327+0000 7fbcbe7fc700 1 -- 172.21.15.123:0/4003773312 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fbcb4084e60 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.328+0000 7fbcc6173700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbcac066f30 0x7fbcac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.328+0000 7fbcc6173700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbcac066f30 0x7fbcac0693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00c5b00 tx=0x7fbcb0009780 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.330+0000 7f12ab7fe700 1 -- 172.21.15.123:0/3494489094 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4005b30 con 0x7f12c406dac0 2023-10-06T04:31:41.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.330+0000 7f12ab7fe700 1 -- 172.21.15.123:0/3494489094 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12b4005c90 con 0x7f12c406dac0 2023-10-06T04:31:41.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.330+0000 7f12ab7fe700 1 -- 172.21.15.123:0/3494489094 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12b4007340 con 0x7f12c406dac0 2023-10-06T04:31:41.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.331+0000 7f0df9d24700 1 -- 172.21.15.123:0/26612400 shutdown_connections 2023-10-06T04:31:41.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.330+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12c4176510 con 0x7f12c406dac0 2023-10-06T04:31:41.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.330+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12c4176ad0 con 0x7f12c406dac0 2023-10-06T04:31:41.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.331+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f12c40c2f00 con 0x7f12c406dac0 2023-10-06T04:31:41.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.333+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcc006c3d0 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.353 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.333+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2dc415c310 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.353 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.334+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2dc415c5f0 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.353 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.335+0000 7f2db27fc700 1 -- 172.21.15.123:0/1622693149 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2db8030bd0 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.336+0000 7f2db27fc700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2da4067080 0x7f2da4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.336+0000 7f2db27fc700 1 -- 172.21.15.123:0/1622693149 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2db8083a10 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.336+0000 7f2dc1d9b700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2da4067080 0x7f2da4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.337+0000 7f2dc1d9b700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2da4067080 0x7f2da4069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4179f00 tx=0x7f2dbc009760 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.336+0000 7f0df9d24700 1 -- 172.21.15.123:0/26612400 wait complete. 2023-10-06T04:31:41.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.337+0000 7f0df9d24700 1 Processor -- start 2023-10-06T04:31:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.338+0000 7f12ab7fe700 1 -- 172.21.15.123:0/3494489094 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f12b4025430 con 0x7f12c406dac0 2023-10-06T04:31:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.338+0000 7f12ab7fe700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f12b0067080 0x7f12b0069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.338+0000 7f12ab7fe700 1 -- 172.21.15.123:0/3494489094 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f12b4084d40 con 0x7f12c406dac0 2023-10-06T04:31:41.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.338+0000 7f12ab7fe700 1 -- 172.21.15.123:0/3494489094 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f12b40851d0 con 0x7f12c406dac0 2023-10-06T04:31:41.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.338+0000 7f0df9d24700 1 -- start start 2023-10-06T04:31:41.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.339+0000 7f0df9d24700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df40c34e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.339+0000 7f0df9d24700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0df4159700 0x7f0df40c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.339+0000 7f0df9d24700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0df40c2080 0x7f0df40c24f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.339+0000 7f0df9d24700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4158de0 con 0x7f0df4159700 2023-10-06T04:31:41.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.339+0000 7f0df9d24700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0df4152c00 con 0x7f0df40c2080 2023-10-06T04:31:41.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.339+0000 7f0df9d24700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0df40c2a30 con 0x7f0df406e960 2023-10-06T04:31:41.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.339+0000 7f12c2ffd700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f12b0067080 0x7f12b0069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.340+0000 7f0df37fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df40c34e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.340+0000 7f0df37fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df40c34e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46516/0 (socket says 172.21.15.123:46516) 2023-10-06T04:31:41.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.340+0000 7f0df37fe700 1 -- 172.21.15.123:0/1117399910 learned_addr learned my addr 172.21.15.123:0/1117399910 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.340+0000 7f12c2ffd700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f12b0067080 0x7f12b0069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12c40c2560 tx=0x7f12b8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0df3fff700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0df40c2080 0x7f0df40c24f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0deaffd700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0df4159700 0x7f0df40c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f2db27fc700 1 -- 172.21.15.123:0/1622693149 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2db8050b80 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7fbcbe7fc700 1 -- 172.21.15.123:0/4003773312 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbcb4051ed0 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.344+0000 7f915fb16700 1 Processor -- start 2023-10-06T04:31:41.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0df37fe700 1 -- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0df40c2080 msgr2=0x7f0df40c24f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0df37fe700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0df40c2080 0x7f0df40c24f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0df37fe700 1 -- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0df4159700 msgr2=0x7f0df40c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0df37fe700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0df4159700 0x7f0df40c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0df37fe700 1 -- 172.21.15.123:0/1117399910 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0dec00bcf0 con 0x7f0df406e960 2023-10-06T04:31:41.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.343+0000 7f0df37fe700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df40c34e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00b5e0 tx=0x7f0ddc009120 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.344+0000 7f915fb16700 1 -- start start 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.345+0000 7f915fb16700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158140e40 0x7f91580b1020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.345+0000 7f915fb16700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580b31c0 con 0x7f9158140a70 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.345+0000 7f915fb16700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91580b3650 con 0x7f91580b1f70 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.345+0000 7f915fb16700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91580b25b0 con 0x7f9158140e40 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.346+0000 7f915d8b2700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9158140a70 0x7f91580aabf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:56980/0 (socket says 172.21.15.123:56980) 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.346+0000 7f915d8b2700 1 -- 172.21.15.123:0/3381469876 learned_addr learned my addr 172.21.15.123:0/3381469876 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.346+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1505946628 0 0) 0x7f91580b31c0 con 0x7f9158140a70 2023-10-06T04:31:41.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.346+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9144003570 con 0x7f9158140a70 2023-10-06T04:31:41.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f0debfff700 1 -- 172.21.15.123:0/1117399910 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00f510 con 0x7f0df406e960 2023-10-06T04:31:41.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f0debfff700 1 -- 172.21.15.123:0/1117399910 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ddc00f670 con 0x7f0df406e960 2023-10-06T04:31:41.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f0debfff700 1 -- 172.21.15.123:0/1117399910 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ddc00fa30 con 0x7f0df406e960 2023-10-06T04:31:41.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0df4158200 con 0x7f0df406e960 2023-10-06T04:31:41.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0df41584b0 con 0x7f0df406e960 2023-10-06T04:31:41.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3772468103 0 0) 0x7f9144003570 con 0x7f9158140a70 2023-10-06T04:31:41.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158140e40 msgr2=0x7f91580b1020 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:31:41.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158140e40 0x7f91580b1020 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 >> v1:172.21.15.177:6789/0 conn(0x7f91580b1f70 legacy=0x7f91580ae970 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:31:41.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580b2890 con 0x7f9158140a70 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f9148001470 con 0x7f9158140a70 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 >> v1:172.21.15.123:6789/0 conn(0x7f9158140a70 legacy=0x7f91580aabf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9144003b10 0x7f9144003f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9144004810 0x7f9144004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91440051c0 0x7f9144014660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580b25b0 con 0x7f9144003b10 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91440056b0 con 0x7f91440051c0 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9144014bf0 con 0x7f9144004810 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.347+0000 7f915d8b2700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9144003b10 0x7f9144003f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915d8b2700 1 -- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9144004810 msgr2=0x7f9144004c80 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915d8b2700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9144004810 0x7f9144004c80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915d8b2700 1 -- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91440051c0 msgr2=0x7f9144014660 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915d8b2700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91440051c0 0x7f9144014660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915d8b2700 1 -- 172.21.15.123:0/3381469876 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580b2890 con 0x7f9144003b10 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915d8b2700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9144003b10 0x7f9144003f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f914800c320 tx=0x7f914800c630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3e957d069cc2c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800ccc0 con 0x7f9144003b10 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800ce20 con 0x7f9144003b10 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.348+0000 7f915c8b0700 1 -- 172.21.15.123:0/3381469876 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9148005b50 con 0x7f9144003b10 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.349+0000 7f0debfff700 1 -- 172.21.15.123:0/1117399910 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ddc01a400 con 0x7f0df406e960 2023-10-06T04:31:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.350+0000 7f0debfff700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dd4067080 0x7f0dd4069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.350+0000 7f0debfff700 1 -- 172.21.15.123:0/1117399910 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0ddc084010 con 0x7f0df406e960 2023-10-06T04:31:41.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.352+0000 7f915fb16700 1 -- 172.21.15.123:0/3381469876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9144003b10 msgr2=0x7f9144003f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.352+0000 7f915fb16700 1 --2- 172.21.15.123:0/3381469876 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9144003b10 0x7f9144003f00 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f914800c320 tx=0x7f914800c630 comp rx=0 tx=0).stop 2023-10-06T04:31:41.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.352+0000 7f915e0b3700 1 -- 172.21.15.123:0/3381469876 reap_dead start 2023-10-06T04:31:41.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.353+0000 7f915fb16700 1 -- 172.21.15.123:0/3381469876 shutdown_connections 2023-10-06T04:31:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.353+0000 7f915fb16700 1 -- 172.21.15.123:0/3381469876 >> 172.21.15.123:0/3381469876 conn(0x7f91580973f0 msgr2=0x7f9158067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.353+0000 7f0deaffd700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dd4067080 0x7f0dd4069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.354+0000 7f0deaffd700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dd4067080 0x7f0dd4069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0df40b2e00 tx=0x7f0de4007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.354+0000 7f915fb16700 1 -- 172.21.15.123:0/3381469876 shutdown_connections 2023-10-06T04:31:41.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.354+0000 7f915fb16700 1 -- 172.21.15.123:0/3381469876 wait complete. 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.356+0000 7f915fb16700 1 Processor -- start 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.356+0000 7f915fb16700 1 -- start start 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915fb16700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9144003b10 0x7f91580ae190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915fb16700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158156c50 0x7f91580ab240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915fb16700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91581599a0 0x7f9158159e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915fb16700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91580ae6d0 con 0x7f91581599a0 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915fb16700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f915815a350 con 0x7f9144003b10 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915fb16700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91580ad0f0 con 0x7f9158156c50 2023-10-06T04:31:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158156c50 0x7f91580ab240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158156c50 0x7f91580ab240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46526/0 (socket says 172.21.15.123:46526) 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 -- 172.21.15.123:0/1843704476 learned_addr learned my addr 172.21.15.123:0/1843704476 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915e0b3700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91581599a0 0x7f9158159e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 -- 172.21.15.123:0/1843704476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9144003b10 msgr2=0x7f91580ae190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9144003b10 0x7f91580ae190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 -- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91581599a0 msgr2=0x7f9158159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91581599a0 0x7f9158159e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.358+0000 7f915d0b1700 1 -- 172.21.15.123:0/1843704476 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91580ad3d0 con 0x7f9158156c50 2023-10-06T04:31:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.359+0000 7f915d0b1700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158156c50 0x7f91580ab240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f915400b4f0 tx=0x7f9154009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.359+0000 7f914dffb700 1 -- 172.21.15.123:0/1843704476 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154031520 con 0x7f9158156c50 2023-10-06T04:31:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.359+0000 7f914dffb700 1 -- 172.21.15.123:0/1843704476 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9154031680 con 0x7f9158156c50 2023-10-06T04:31:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.359+0000 7f914dffb700 1 -- 172.21.15.123:0/1843704476 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9154031a40 con 0x7f9158156c50 2023-10-06T04:31:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.360+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91580ad6b0 con 0x7f9158156c50 2023-10-06T04:31:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.360+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91580adc70 con 0x7f9158156c50 2023-10-06T04:31:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.361+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91580ade90 con 0x7f9158156c50 2023-10-06T04:31:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.362+0000 7f914dffb700 1 -- 172.21.15.123:0/1843704476 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9154033410 con 0x7f9158156c50 2023-10-06T04:31:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.363+0000 7f914dffb700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f913c067080 0x7f913c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.363+0000 7f914dffb700 1 -- 172.21.15.123:0/1843704476 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f91540a5f50 con 0x7f9158156c50 2023-10-06T04:31:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.364+0000 7f915d8b2700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f913c067080 0x7f913c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.364+0000 7f915d8b2700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f913c067080 0x7f913c069530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9148000c00 tx=0x7f9148015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.364+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0dd80052b0 con 0x7f0df406e960 2023-10-06T04:31:41.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.367+0000 7f914dffb700 1 -- 172.21.15.123:0/1843704476 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9154073180 con 0x7f9158156c50 2023-10-06T04:31:41.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.372+0000 7f0debfff700 1 -- 172.21.15.123:0/1117399910 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ddc051180 con 0x7f0df406e960 2023-10-06T04:31:41.647 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.645+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f6da00c6490 con 0x7f6da0155530 2023-10-06T04:31:41.650 INFO:teuthology.orchestra.run.smithi123.stdout:55834574854 2023-10-06T04:31:41.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.646+0000 7f6d95ffb700 1 -- 172.21.15.123:0/377103829 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6d98057b90 con 0x7f6da0155530 2023-10-06T04:31:41.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6d8c0671a0 msgr2=0x7f6d8c069650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6d8c0671a0 0x7f6d8c069650 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c007fd0 tx=0x7f6d9c007a50 comp rx=0 tx=0).stop 2023-10-06T04:31:41.652 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6da0155530 msgr2=0x7f6da00c1d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.652 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6da0155530 0x7f6da00c1d20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f6d980129a0 tx=0x7f6d98018000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 shutdown_connections 2023-10-06T04:31:41.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6d8c0671a0 0x7f6d8c069650 secure :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f6d9c007fd0 tx=0x7f6d9c007a50 comp rx=0 tx=0).stop 2023-10-06T04:31:41.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6da0155530 0x7f6da00c1d20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6da006dac0 0x7f6da0148c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 --2- 172.21.15.123:0/377103829 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6da0155160 0x7f6da016d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 >> 172.21.15.123:0/377103829 conn(0x7f6da0067660 msgr2=0x7f6da014bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.650+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 shutdown_connections 2023-10-06T04:31:41.654 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.651+0000 7f6da8220700 1 -- 172.21.15.123:0/377103829 wait complete. 2023-10-06T04:31:41.670 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.2 2023-10-06T04:31:41.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.690+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f4ed80c82b0 con 0x7f4ec4004810 2023-10-06T04:31:41.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.691+0000 7f4ecdffb700 1 -- 172.21.15.123:0/3379555403 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f4ec8089090 con 0x7f4ec4004810 2023-10-06T04:31:41.692 INFO:teuthology.orchestra.run.smithi123.stdout:55834574854 2023-10-06T04:31:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4ebc067000 msgr2=0x7f4ebc0694b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4ebc067000 0x7f4ebc0694b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001500 tx=0x7f4ed4015000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4004810 msgr2=0x7f4ed80c6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4004810 0x7f4ed80c6950 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f4ed804d2d0 tx=0x7f4ec8008040 comp rx=0 tx=0).stop 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 shutdown_connections 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4ebc067000 0x7f4ebc0694b0 secure :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4001500 tx=0x7f4ed4015000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4ec4004810 0x7f4ed80c6950 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4ed8177cc0 0x7f4ed80b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 --2- 172.21.15.123:0/3379555403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4ed8177850 0x7f4ed80b1280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.694+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 >> 172.21.15.123:0/3379555403 conn(0x7f4ed80680f0 msgr2=0x7f4ed80c3bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.695+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 shutdown_connections 2023-10-06T04:31:41.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.695+0000 7f4edfc40700 1 -- 172.21.15.123:0/3379555403 wait complete. 2023-10-06T04:31:41.715 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.3 2023-10-06T04:31:41.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.715+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f02000c5180 con 0x7f0200151ca0 2023-10-06T04:31:41.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.716+0000 7f01f67fc700 1 -- 172.21.15.123:0/4268418006 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f02000c5180 con 0x7f0200151ca0 2023-10-06T04:31:41.717 INFO:teuthology.orchestra.run.smithi123.stdout:55834574854 2023-10-06T04:31:41.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.718+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f0dd8005560 con 0x7f0df406e960 2023-10-06T04:31:41.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.719+0000 7f0debfff700 1 -- 172.21.15.123:0/1117399910 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f0ddc0547a0 con 0x7f0df406e960 2023-10-06T04:31:41.720 INFO:teuthology.orchestra.run.smithi123.stdout:55834574854 2023-10-06T04:31:41.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.720+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01e4067080 msgr2=0x7f01e4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.720+0000 7f0208761700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01e4067080 0x7f01e4069530 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f01f0006fd0 tx=0x7f01f000b040 comp rx=0 tx=0).stop 2023-10-06T04:31:41.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.720+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200151ca0 msgr2=0x7f02000afbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.720+0000 7f0208761700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200151ca0 0x7f02000afbb0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f0200071110 tx=0x7f01fc00b320 comp rx=0 tx=0).stop 2023-10-06T04:31:41.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.720+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fbcc00c3f70 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.722 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.720+0000 7fbcbe7fc700 1 -- 172.21.15.123:0/4003773312 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbcb401d030 con 0x7fbcc00c8dd0 2023-10-06T04:31:41.722 INFO:teuthology.orchestra.run.smithi123.stdout:55834574853 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dd4067080 msgr2=0x7f0dd4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dd4067080 0x7f0dd4069530 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f0df40b2e00 tx=0x7f0de4007ea0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 msgr2=0x7f0df40c34e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df40c34e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f0ddc00b5e0 tx=0x7f0ddc009120 comp rx=0 tx=0).stop 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 shutdown_connections 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dd4067080 0x7f0dd4069530 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0df40c2080 0x7f0df40c24f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0df406e960 0x7f0df40c34e0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 --2- 172.21.15.123:0/1117399910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0df4159700 0x7f0df40c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 >> 172.21.15.123:0/1117399910 conn(0x7f0df40cd010 msgr2=0x7f0df40c7fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 shutdown_connections 2023-10-06T04:31:41.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.721+0000 7f0df9d24700 1 -- 172.21.15.123:0/1117399910 wait complete. 2023-10-06T04:31:41.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.722+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbcac066f30 msgr2=0x7fbcac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.722+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbcac066f30 0x7fbcac0693e0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7fbcc00c5b00 tx=0x7fbcb0009780 comp rx=0 tx=0).stop 2023-10-06T04:31:41.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.722+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 msgr2=0x7fbcc0177430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.722+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc0177430 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7fbcb400d990 tx=0x7fbcb4006090 comp rx=0 tx=0).stop 2023-10-06T04:31:41.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.723+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 shutdown_connections 2023-10-06T04:31:41.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.723+0000 7f0208761700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f01e4067080 0x7f01e4069530 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.723+0000 7f0208761700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01ec0051c0 0x7f02001616a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.723+0000 7f0208761700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f02000b0540 0x7f02000c5460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.723+0000 7f0208761700 1 --2- 172.21.15.123:0/4268418006 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0200151ca0 0x7f02000afbb0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.723+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 >> 172.21.15.123:0/4268418006 conn(0x7f0200068130 msgr2=0x7f02001548e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 shutdown_connections 2023-10-06T04:31:41.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbcac066f30 0x7fbcac0693e0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbcc0155160 0x7fbcc0179860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbcc006dac0 0x7fbcc01701b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 --2- 172.21.15.123:0/4003773312 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbcc00c8dd0 0x7fbcc0177430 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 >> 172.21.15.123:0/4003773312 conn(0x7fbcc0067660 msgr2=0x7fbcc00ac820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f2dc40c9b30 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 shutdown_connections 2023-10-06T04:31:41.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.725+0000 7fbcc83d7700 1 -- 172.21.15.123:0/4003773312 wait complete. 2023-10-06T04:31:41.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.726+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 shutdown_connections 2023-10-06T04:31:41.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.726+0000 7f0208761700 1 -- 172.21.15.123:0/4268418006 wait complete. 2023-10-06T04:31:41.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.727+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f12c40c6490 con 0x7f12c406dac0 2023-10-06T04:31:41.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.727+0000 7f2db27fc700 1 -- 172.21.15.123:0/1622693149 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2db80541a0 con 0x7f2dc40c6bd0 2023-10-06T04:31:41.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.731+0000 7f12ab7fe700 1 -- 172.21.15.123:0/3494489094 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f12b4051f60 con 0x7f12c406dac0 2023-10-06T04:31:41.732 INFO:teuthology.orchestra.run.smithi123.stdout:55834574854 2023-10-06T04:31:41.733 INFO:teuthology.orchestra.run.smithi123.stdout:55834574854 2023-10-06T04:31:41.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f12b0067080 msgr2=0x7f12b0069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f12b0067080 0x7f12b0069530 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12c40c2560 tx=0x7f12b8008040 comp rx=0 tx=0).stop 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c406dac0 msgr2=0x7f12c40a5770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c406dac0 0x7f12c40a5770 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f12b400c7f0 tx=0x7f12b4007e00 comp rx=0 tx=0).stop 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 shutdown_connections 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f12b0067080 0x7f12b0069530 secure :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f12c40c2560 tx=0x7f12b8008040 comp rx=0 tx=0).stop 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f12c406dac0 0x7f12c40a5770 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f12c4155530 0x7f12c4175da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 --2- 172.21.15.123:0/3494489094 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f12c4155160 0x7f12c40c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.734+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 >> 172.21.15.123:0/3494489094 conn(0x7f12c4067660 msgr2=0x7f12c40b06f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.738 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.7 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2da4067080 msgr2=0x7f2da4069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2da4067080 0x7f2da4069530 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f2dc4179f00 tx=0x7f2dbc009760 comp rx=0 tx=0).stop 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2dc40c6bd0 msgr2=0x7f2dc40c4a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2dc40c6bd0 0x7f2dc40c4a10 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f2db800b630 tx=0x7f2db80091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 shutdown_connections 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2da4067080 0x7f2da4069530 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2dc415da00 0x7f2dc417a4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2dc40c6bd0 0x7f2dc40c4a10 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 --2- 172.21.15.123:0/1622693149 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2dc40c9f10 0x7f2dc4179fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 >> 172.21.15.123:0/1622693149 conn(0x7f2dc4068130 msgr2=0x7f2dc406ec90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 shutdown_connections 2023-10-06T04:31:41.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.735+0000 7f2dc8caf700 1 -- 172.21.15.123:0/1622693149 wait complete. 2023-10-06T04:31:41.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.737+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 shutdown_connections 2023-10-06T04:31:41.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.737+0000 7f12c9c0f700 1 -- 172.21.15.123:0/3494489094 wait complete. 2023-10-06T04:31:41.745 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574853 for osd.6 2023-10-06T04:31:41.751 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.5 2023-10-06T04:31:41.752 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.0 2023-10-06T04:31:41.753 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.1 2023-10-06T04:31:41.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.753+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f915806d530 con 0x7f9158156c50 2023-10-06T04:31:41.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.753+0000 7f914dffb700 1 -- 172.21.15.123:0/1843704476 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f91540767a0 con 0x7f9158156c50 2023-10-06T04:31:41.754 INFO:teuthology.orchestra.run.smithi123.stdout:55834574855 2023-10-06T04:31:41.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f913c067080 msgr2=0x7f913c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f913c067080 0x7f913c069530 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f9148000c00 tx=0x7f9148015000 comp rx=0 tx=0).stop 2023-10-06T04:31:41.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158156c50 msgr2=0x7f91580ab240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:41.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158156c50 0x7f91580ab240 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f915400b4f0 tx=0x7f9154009060 comp rx=0 tx=0).stop 2023-10-06T04:31:41.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 shutdown_connections 2023-10-06T04:31:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f913c067080 0x7f913c069530 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f91581599a0 0x7f9158159e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9144003b10 0x7f91580ae190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.755+0000 7f915fb16700 1 --2- 172.21.15.123:0/1843704476 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9158156c50 0x7f91580ab240 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.756+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 >> 172.21.15.123:0/1843704476 conn(0x7f91580973f0 msgr2=0x7f915806cab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.756+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 shutdown_connections 2023-10-06T04:31:41.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:41.756+0000 7f915fb16700 1 -- 172.21.15.123:0/1843704476 wait complete. 2023-10-06T04:31:41.769 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574855 for osd.4 2023-10-06T04:31:42.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T04:31:42.716 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-10-06T04:31:42.739 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-10-06T04:31:42.745 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-10-06T04:31:42.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-10-06T04:31:42.753 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-10-06T04:31:42.754 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-10-06T04:31:42.770 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-10-06T04:31:42.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.841+0000 7fd2acdae700 1 Processor -- start 2023-10-06T04:31:42.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.841+0000 7fd2acdae700 1 -- start start 2023-10-06T04:31:42.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.842+0000 7fd2acdae700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd2a8155160 0x7fd2a80691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.842+0000 7fd2acdae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd2a806dac0 0x7fd2a80c86c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.842+0000 7fd2acdae700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2a80c6800 con 0x7fd2a8155530 2023-10-06T04:31:42.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.842+0000 7fd2acdae700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a8069a00 con 0x7fd2a8155160 2023-10-06T04:31:42.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.842+0000 7fd2acdae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2a8069ce0 con 0x7fd2a806dac0 2023-10-06T04:31:42.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.843+0000 7fd2a6d9d700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fd2a8155530 0x7fd2a8158ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:53354/0 (socket says 172.21.15.123:53354) 2023-10-06T04:31:42.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.843+0000 7fd2a6d9d700 1 -- 172.21.15.123:0/3048418831 learned_addr learned my addr 172.21.15.123:0/3048418831 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:42.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.850+0000 7fd2a5d9b700 1 --2- 172.21.15.123:0/3048418831 >> v2:172.21.15.177:3300/0 conn(0x7fd2a806dac0 0x7fd2a80c86c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.850+0000 7fd2a659c700 1 --2- 172.21.15.123:0/3048418831 >> v2:172.21.15.123:3300/0 conn(0x7fd2a8155160 0x7fd2a80691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a559a700 1 -- 172.21.15.123:0/3048418831 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 312355677 0 0) 0x7fd2a80c6800 con 0x7fd2a8155530 2023-10-06T04:31:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a559a700 1 -- 172.21.15.123:0/3048418831 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd290003570 con 0x7fd2a8155530 2023-10-06T04:31:42.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a5d9b700 1 -- 172.21.15.123:0/3048418831 >> v2:172.21.15.123:3300/0 conn(0x7fd2a8155160 msgr2=0x7fd2a80691b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a5d9b700 1 --2- 172.21.15.123:0/3048418831 >> v2:172.21.15.123:3300/0 conn(0x7fd2a8155160 0x7fd2a80691b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a5d9b700 1 -- 172.21.15.123:0/3048418831 >> v1:172.21.15.177:6789/0 conn(0x7fd2a8155530 legacy=0x7fd2a8158ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a5d9b700 1 -- 172.21.15.123:0/3048418831 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2a8069fc0 con 0x7fd2a806dac0 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a659c700 1 --2- 172.21.15.123:0/3048418831 >> v2:172.21.15.123:3300/0 conn(0x7fd2a8155160 0x7fd2a80691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.855+0000 7fd2a5d9b700 1 --2- 172.21.15.123:0/3048418831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a806dac0 0x7fd2a80c86c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0098e0 tx=0x7fd29c0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e979cc4ad0dea0b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.856+0000 7fd2a559a700 1 -- 172.21.15.123:0/3048418831 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c011630 con 0x7fd2a806dac0 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.856+0000 7fd2a559a700 1 -- 172.21.15.123:0/3048418831 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd29c011790 con 0x7fd2a806dac0 2023-10-06T04:31:42.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.856+0000 7fd2a559a700 1 -- 172.21.15.123:0/3048418831 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd29c011b50 con 0x7fd2a806dac0 2023-10-06T04:31:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 -- 172.21.15.123:0/3048418831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a806dac0 msgr2=0x7fd2a80c86c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3048418831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a806dac0 0x7fd2a80c86c0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fd29c0098e0 tx=0x7fd29c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 -- 172.21.15.123:0/3048418831 shutdown_connections 2023-10-06T04:31:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3048418831 >> v2:172.21.15.123:3300/0 conn(0x7fd2a8155160 0x7fd2a80691b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3048418831 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a806dac0 0x7fd2a80c86c0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 -- 172.21.15.123:0/3048418831 >> 172.21.15.123:0/3048418831 conn(0x7fd2a8067660 msgr2=0x7fd2a8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 -- 172.21.15.123:0/3048418831 shutdown_connections 2023-10-06T04:31:42.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.857+0000 7fd2acdae700 1 -- 172.21.15.123:0/3048418831 wait complete. 2023-10-06T04:31:42.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.859+0000 7fd2acdae700 1 Processor -- start 2023-10-06T04:31:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.859+0000 7fd2acdae700 1 -- start start 2023-10-06T04:31:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.860+0000 7fd2acdae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2a806dac0 0x7fd2a8152b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.860+0000 7fd2acdae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8155160 0x7fd2a8150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.860+0000 7fd2acdae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a8155530 0x7fd2a8151270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.860+0000 7fd2acdae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8152670 con 0x7fd2a8155160 2023-10-06T04:31:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.860+0000 7fd2acdae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a81517b0 con 0x7fd2a8155530 2023-10-06T04:31:42.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.860+0000 7fd2acdae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2a8151a60 con 0x7fd2a806dac0 2023-10-06T04:31:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.861+0000 7fd2a659c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2a806dac0 0x7fd2a8152b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.861+0000 7fd2a6d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a8155530 0x7fd2a8151270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.861+0000 7fd2a6d9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a8155530 0x7fd2a8151270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41194/0 (socket says 172.21.15.123:41194) 2023-10-06T04:31:42.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.861+0000 7fd2a6d9d700 1 -- 172.21.15.123:0/3876135962 learned_addr learned my addr 172.21.15.123:0/3876135962 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.861+0000 7fd2a5d9b700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8155160 0x7fd2a8150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.861+0000 7fd2a6d9d700 1 -- 172.21.15.123:0/3876135962 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2a806dac0 msgr2=0x7fd2a8152b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.861+0000 7fd2a6d9d700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2a806dac0 0x7fd2a8152b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.862+0000 7fd2a6d9d700 1 -- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8155160 msgr2=0x7fd2a8150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.862+0000 7fd2a6d9d700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8155160 0x7fd2a8150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.862+0000 7fd2a6d9d700 1 -- 172.21.15.123:0/3876135962 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29c009420 con 0x7fd2a8155530 2023-10-06T04:31:42.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.862+0000 7fd2a6d9d700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a8155530 0x7fd2a8151270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0001bb0 tx=0x7fd2a0006180 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.865+0000 7fd2967fc700 1 -- 172.21.15.123:0/3876135962 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00082e0 con 0x7fd2a8155530 2023-10-06T04:31:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.865+0000 7fd2967fc700 1 -- 172.21.15.123:0/3876135962 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2a0006510 con 0x7fd2a8155530 2023-10-06T04:31:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.865+0000 7fd2967fc700 1 -- 172.21.15.123:0/3876135962 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2a00068d0 con 0x7fd2a8155530 2023-10-06T04:31:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.865+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2a8151d40 con 0x7fd2a8155530 2023-10-06T04:31:42.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.866+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2a8152300 con 0x7fd2a8155530 2023-10-06T04:31:42.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.871+0000 7fd2967fc700 1 -- 172.21.15.123:0/3876135962 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd2a0006c80 con 0x7fd2a8155530 2023-10-06T04:31:42.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.872+0000 7fd2967fc700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd288067080 0x7fd288069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.872+0000 7fd2967fc700 1 -- 172.21.15.123:0/3876135962 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd2a0083e30 con 0x7fd2a8155530 2023-10-06T04:31:42.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.877+0000 7fd2a659c700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd288067080 0x7fd288069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.878+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd28c0052b0 con 0x7fd2a8155530 2023-10-06T04:31:42.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.879+0000 7fd2a659c700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd288067080 0x7fd288069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd29800bc90 tx=0x7fd298008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.883+0000 7fd2967fc700 1 -- 172.21.15.123:0/3876135962 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd2a0050f20 con 0x7fd2a8155530 2023-10-06T04:31:42.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.889+0000 7fb0943db700 1 Processor -- start 2023-10-06T04:31:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.890+0000 7fb0943db700 1 -- start start 2023-10-06T04:31:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.891+0000 7fb0943db700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb08c0bf330 0x7fb08c0691b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.891+0000 7fb0943db700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c0696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.891+0000 7fb0943db700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb08c0c2f00 con 0x7fb08c0c3390 2023-10-06T04:31:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.891+0000 7fb0943db700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb08c0c7cf0 con 0x7fb08c0bf330 2023-10-06T04:31:42.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.891+0000 7fb0943db700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0c7fd0 con 0x7fb08c0cca10 2023-10-06T04:31:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.892+0000 7fb091976700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c0696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.892+0000 7fb091976700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c0696f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46546/0 (socket says 172.21.15.123:46546) 2023-10-06T04:31:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.892+0000 7fb091976700 1 -- 172.21.15.123:0/2956539345 learned_addr learned my addr 172.21.15.123:0/2956539345 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.892+0000 7fb092177700 1 --2- 172.21.15.123:0/2956539345 >> v2:172.21.15.177:3300/0 conn(0x7fb08c0bf330 0x7fb08c0691b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.895 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.893+0000 7fb092177700 1 -- 172.21.15.123:0/2956539345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 msgr2=0x7fb08c0696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.893+0000 7fb092177700 1 --2- 172.21.15.123:0/2956539345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c0696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.893+0000 7fb092177700 1 -- 172.21.15.123:0/2956539345 >> v1:172.21.15.177:6789/0 conn(0x7fb08c0c3390 legacy=0x7fb08c0c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.893+0000 7fb092177700 1 -- 172.21.15.123:0/2956539345 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb08c0c82b0 con 0x7fb08c0bf330 2023-10-06T04:31:42.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.893+0000 7fb092177700 1 --2- 172.21.15.123:0/2956539345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0bf330 0x7fb08c0691b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0880098e0 tx=0x7fb0880091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a9d1c859f02cb366 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.894+0000 7fb091175700 1 -- 172.21.15.123:0/2956539345 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800b640 con 0x7fb08c0bf330 2023-10-06T04:31:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.894+0000 7fb091175700 1 -- 172.21.15.123:0/2956539345 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb08800b7a0 con 0x7fb08c0bf330 2023-10-06T04:31:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.894+0000 7fb091175700 1 -- 172.21.15.123:0/2956539345 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb08800bb60 con 0x7fb08c0bf330 2023-10-06T04:31:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 -- 172.21.15.123:0/2956539345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0bf330 msgr2=0x7fb08c0691b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 --2- 172.21.15.123:0/2956539345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0bf330 0x7fb08c0691b0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fb0880098e0 tx=0x7fb0880091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 -- 172.21.15.123:0/2956539345 shutdown_connections 2023-10-06T04:31:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 --2- 172.21.15.123:0/2956539345 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0bf330 0x7fb08c0691b0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 --2- 172.21.15.123:0/2956539345 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c0696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 -- 172.21.15.123:0/2956539345 >> 172.21.15.123:0/2956539345 conn(0x7fb08c0680f0 msgr2=0x7fb08c0cbef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:42.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 -- 172.21.15.123:0/2956539345 shutdown_connections 2023-10-06T04:31:42.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.895+0000 7fb0943db700 1 -- 172.21.15.123:0/2956539345 wait complete. 2023-10-06T04:31:42.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.897+0000 7fb0943db700 1 Processor -- start 2023-10-06T04:31:42.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.898+0000 7fb0943db700 1 -- start start 2023-10-06T04:31:42.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.899+0000 7fb0943db700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08c0bf330 0x7fb08c0b0c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.899+0000 7fb0943db700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0c3390 0x7fb08c14df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.899+0000 7fb0943db700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c143a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.899+0000 7fb0943db700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c0b1950 con 0x7fb08c0bf330 2023-10-06T04:31:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.899+0000 7fb0943db700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c15e640 con 0x7fb08c0c3390 2023-10-06T04:31:42.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.899+0000 7fb0943db700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb08c15e030 con 0x7fb08c0cca10 2023-10-06T04:31:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.899+0000 7fb091976700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0c3390 0x7fb08c14df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c143a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c143a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46560/0 (socket says 172.21.15.123:46560) 2023-10-06T04:31:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 -- 172.21.15.123:0/630831131 learned_addr learned my addr 172.21.15.123:0/630831131 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 -- 172.21.15.123:0/630831131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0c3390 msgr2=0x7fb08c14df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0c3390 0x7fb08c14df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 -- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08c0bf330 msgr2=0x7fb08c0b0c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08c0bf330 0x7fb08c0b0c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.900+0000 7fb092978700 1 -- 172.21.15.123:0/630831131 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb088009420 con 0x7fb08c0cca10 2023-10-06T04:31:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.901+0000 7fb092978700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c143a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb08400c940 tx=0x7fb0840016d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.902+0000 7fb0827fc700 1 -- 172.21.15.123:0/630831131 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084012630 con 0x7fb08c0cca10 2023-10-06T04:31:42.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.902+0000 7fb0827fc700 1 -- 172.21.15.123:0/630831131 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0840127b0 con 0x7fb08c0cca10 2023-10-06T04:31:42.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.902+0000 7fb0827fc700 1 -- 172.21.15.123:0/630831131 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb084012b70 con 0x7fb08c0cca10 2023-10-06T04:31:42.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.902+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb08c0b1670 con 0x7fb08c0cca10 2023-10-06T04:31:42.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.902+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb08c14d3b0 con 0x7fb08c0cca10 2023-10-06T04:31:42.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.904+0000 7fb0827fc700 1 -- 172.21.15.123:0/630831131 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb084003030 con 0x7fb08c0cca10 2023-10-06T04:31:42.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.905+0000 7fb0827fc700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb070067080 0x7fb070069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.905+0000 7fb0827fc700 1 -- 172.21.15.123:0/630831131 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb084084560 con 0x7fb08c0cca10 2023-10-06T04:31:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.906+0000 7fb092177700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb070067080 0x7fb070069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.906+0000 7fb092177700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb070067080 0x7fb070069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb08800a010 tx=0x7fb088017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.909+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0740052b0 con 0x7fb08c0cca10 2023-10-06T04:31:42.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.914+0000 7fb0827fc700 1 -- 172.21.15.123:0/630831131 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb084051650 con 0x7fb08c0cca10 2023-10-06T04:31:42.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.959+0000 7effb9e2d700 1 Processor -- start 2023-10-06T04:31:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.959+0000 7f1deeb3d700 1 Processor -- start 2023-10-06T04:31:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.959+0000 7f1deeb3d700 1 -- start start 2023-10-06T04:31:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.960+0000 7f1deeb3d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1de815d5c0 0x7f1de80c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.960+0000 7f1deeb3d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de80c7ae0 con 0x7f1de806f180 2023-10-06T04:31:42.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.960+0000 7f1deeb3d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de80c6ed0 con 0x7f1de8161aa0 2023-10-06T04:31:42.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.960+0000 7f1deeb3d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de80c71b0 con 0x7f1de815d5c0 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.960+0000 7effb9e2d700 1 -- start start 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7effb9e2d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effb40688c0 0x7effb4155430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7effb9e2d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb40c6dd0 con 0x7effb406a5d0 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7effb9e2d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effb40cc3e0 con 0x7effb4068d60 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7effb9e2d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effb4063940 con 0x7effb40688c0 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7effabfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effb40688c0 0x7effb4155430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7effabfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effb40688c0 0x7effb4155430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43402/0 (socket says 172.21.15.123:43402) 2023-10-06T04:31:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7effabfff700 1 -- 172.21.15.123:0/3581943037 learned_addr learned my addr 172.21.15.123:0/3581943037 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1dec8d9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1de815d5c0 0x7f1de80c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1dec8d9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1de815d5c0 0x7f1de80c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43398/0 (socket says 172.21.15.123:43398) 2023-10-06T04:31:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1dec8d9700 1 -- 172.21.15.123:0/2634475650 learned_addr learned my addr 172.21.15.123:0/2634475650 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1de77fe700 1 -- 172.21.15.123:0/2634475650 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2823483847 0 0) 0x7f1de80c6ed0 con 0x7f1de8161aa0 2023-10-06T04:31:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1de77fe700 1 -- 172.21.15.123:0/2634475650 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd8003570 con 0x7f1de8161aa0 2023-10-06T04:31:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1dec8d9700 1 -- 172.21.15.123:0/2634475650 >> v1:172.21.15.177:6789/0 conn(0x7f1de8161aa0 legacy=0x7f1de80c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1de77fe700 1 -- 172.21.15.123:0/2634475650 <== mon.1 v1:172.21.15.177:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 756983927 0 0) 0x7f1dd8003570 con 0x7f1de8161aa0 2023-10-06T04:31:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1dec8d9700 1 -- 172.21.15.123:0/2634475650 >> v1:172.21.15.123:6789/0 conn(0x7f1de806f180 legacy=0x7f1de8161390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.961+0000 7f1dec8d9700 1 -- 172.21.15.123:0/2634475650 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de80c7490 con 0x7f1de815d5c0 2023-10-06T04:31:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7f1dec8d9700 1 --2- 172.21.15.123:0/2634475650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de815d5c0 0x7f1de80c66a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009930 tx=0x7f1dd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9477f8164ea39dd7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7effabfff700 1 -- 172.21.15.123:0/3581943037 >> v1:172.21.15.177:6789/0 conn(0x7effb4068d60 legacy=0x7effb4155970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:31:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7effabfff700 1 -- 172.21.15.123:0/3581943037 >> v1:172.21.15.123:6789/0 conn(0x7effb406a5d0 legacy=0x7effb4063230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7f5880ce4700 1 Processor -- start 2023-10-06T04:31:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7effabfff700 1 -- 172.21.15.123:0/3581943037 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effb40c7070 con 0x7effb40688c0 2023-10-06T04:31:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7f1de77fe700 1 -- 172.21.15.123:0/2634475650 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1de00017f0 con 0x7f1de815d5c0 2023-10-06T04:31:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7f1de77fe700 1 -- 172.21.15.123:0/2634475650 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4011420 con 0x7f1de815d5c0 2023-10-06T04:31:42.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7f5880ce4700 1 -- start start 2023-10-06T04:31:42.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.962+0000 7f1de77fe700 1 -- 172.21.15.123:0/2634475650 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd40117a0 con 0x7f1de815d5c0 2023-10-06T04:31:42.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.963+0000 7effabfff700 1 --2- 172.21.15.123:0/3581943037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb40688c0 0x7effb4155430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0099b0 tx=0x7eff9c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22b3bf5482de32d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.963+0000 7f1deeb3d700 1 -- 172.21.15.123:0/2634475650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de815d5c0 msgr2=0x7f1de80c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.963+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/2634475650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de815d5c0 0x7f1de80c66a0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f1dd4009930 tx=0x7f1dd4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.963+0000 7f039b59e700 1 Processor -- start 2023-10-06T04:31:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.963+0000 7f039b59e700 1 -- start start 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f5880ce4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c069ea0 0x7f587c0c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f5880ce4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f587c0c8bc0 0x7f587c06e190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f5880ce4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587c06d130 con 0x7f587c15c880 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f5880ce4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587c06d3d0 con 0x7f587c0c8bc0 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f5880ce4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c06d6b0 con 0x7f587c069ea0 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587a59c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f587c15c880 0x7f587c0c6a50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:53388/0 (socket says 172.21.15.123:53388) 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587a59c700 1 -- 172.21.15.123:0/311127642 learned_addr learned my addr 172.21.15.123:0/311127642 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587ad9d700 1 --2- 172.21.15.123:0/311127642 >> v2:172.21.15.177:3300/0 conn(0x7f587c0c8bc0 0x7f587c06e190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587959a700 1 -- 172.21.15.123:0/311127642 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1720163992 0 0) 0x7f587c06d130 con 0x7f587c15c880 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587959a700 1 -- 172.21.15.123:0/311127642 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5864003570 con 0x7f587c15c880 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587ad9d700 1 -- 172.21.15.123:0/311127642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c069ea0 msgr2=0x7f587c0c8680 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587ad9d700 1 --2- 172.21.15.123:0/311127642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c069ea0 0x7f587c0c8680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587ad9d700 1 -- 172.21.15.123:0/311127642 >> v1:172.21.15.177:6789/0 conn(0x7f587c15c880 legacy=0x7f587c0c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f587ad9d700 1 -- 172.21.15.123:0/311127642 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587c06d990 con 0x7f587c0c8bc0 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7effb2ffd700 1 -- 172.21.15.123:0/3581943037 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00b760 con 0x7effb40688c0 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7effb2ffd700 1 -- 172.21.15.123:0/3581943037 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff9c00b8c0 con 0x7effb40688c0 2023-10-06T04:31:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7effb2ffd700 1 -- 172.21.15.123:0/3581943037 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9c00bc60 con 0x7effb40688c0 2023-10-06T04:31:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f039b59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c06d860 0x7f039c06dc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.964+0000 7f039b59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f039c1612e0 0x7f039c161730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f039b59e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f039c161c70 con 0x7f039c0c9f60 2023-10-06T04:31:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f039b59e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f039c0cb480 con 0x7f039c1612e0 2023-10-06T04:31:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f039b59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c06e170 con 0x7f039c06d860 2023-10-06T04:31:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f587ad9d700 1 --2- 172.21.15.123:0/311127642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c06e190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5870009940 tx=0x7f58700091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b224ebec637923b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f587959a700 1 -- 172.21.15.123:0/311127642 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870022640 con 0x7f587c0c8bc0 2023-10-06T04:31:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f587959a700 1 -- 172.21.15.123:0/311127642 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58700227a0 con 0x7f587c0c8bc0 2023-10-06T04:31:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f587959a700 1 -- 172.21.15.123:0/311127642 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5870022b60 con 0x7f587c0c8bc0 2023-10-06T04:31:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7effb9e2d700 1 -- 172.21.15.123:0/3581943037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb40688c0 msgr2=0x7effb4155430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7effb9e2d700 1 --2- 172.21.15.123:0/3581943037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb40688c0 0x7effb4155430 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0099b0 tx=0x7eff9c009250 comp rx=0 tx=0).stop 2023-10-06T04:31:42.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f1deeb3d700 1 -- 172.21.15.123:0/2634475650 shutdown_connections 2023-10-06T04:31:42.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/2634475650 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de815d5c0 0x7f1de80c66a0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f1deeb3d700 1 -- 172.21.15.123:0/2634475650 >> 172.21.15.123:0/2634475650 conn(0x7f1de8067660 msgr2=0x7f1de8067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:42.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7effb9e2d700 1 -- 172.21.15.123:0/3581943037 shutdown_connections 2023-10-06T04:31:42.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7effb9e2d700 1 --2- 172.21.15.123:0/3581943037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb40688c0 0x7effb4155430 secure :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0099b0 tx=0x7eff9c009250 comp rx=0 tx=0).stop 2023-10-06T04:31:42.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7effb9e2d700 1 -- 172.21.15.123:0/3581943037 >> 172.21.15.123:0/3581943037 conn(0x7effb40c9d00 msgr2=0x7effb406da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:42.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f5880ce4700 1 -- 172.21.15.123:0/311127642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 msgr2=0x7f587c06e190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f5880ce4700 1 --2- 172.21.15.123:0/311127642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c06e190 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f5870009940 tx=0x7f58700091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f039c1612e0 0x7f039c161730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f039c1612e0 0x7f039c161730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43404/0 (socket says 172.21.15.123:43404) 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 -- 172.21.15.123:0/1682315446 learned_addr learned my addr 172.21.15.123:0/1682315446 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f039a59c700 1 --2- 172.21.15.123:0/1682315446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c06d860 0x7f039c06dc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 -- 172.21.15.123:0/1682315446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c06d860 msgr2=0x7f039c06dc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 --2- 172.21.15.123:0/1682315446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c06d860 0x7f039c06dc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 -- 172.21.15.123:0/1682315446 >> v1:172.21.15.177:6789/0 conn(0x7f039c0c9f60 legacy=0x7f039c0ca3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 -- 172.21.15.123:0/1682315446 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f039c0cabe0 con 0x7f039c1612e0 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.965+0000 7f0399d9b700 1 --2- 172.21.15.123:0/1682315446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c1612e0 0x7f039c161730 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f038c0099b0 tx=0x7f038c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40970a42f5ea4792 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f039959a700 1 -- 172.21.15.123:0/1682315446 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c009ea0 con 0x7f039c1612e0 2023-10-06T04:31:42.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f039959a700 1 -- 172.21.15.123:0/1682315446 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038c00b820 con 0x7f039c1612e0 2023-10-06T04:31:42.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7f039959a700 1 -- 172.21.15.123:0/1682315446 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038c00bbc0 con 0x7f039c1612e0 2023-10-06T04:31:42.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.966+0000 7effb9e2d700 1 -- 172.21.15.123:0/3581943037 shutdown_connections 2023-10-06T04:31:42.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.967+0000 7effb9e2d700 1 -- 172.21.15.123:0/3581943037 wait complete. 2023-10-06T04:31:42.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.967+0000 7f039b59e700 1 -- 172.21.15.123:0/1682315446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c1612e0 msgr2=0x7f039c161730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.967+0000 7f039b59e700 1 --2- 172.21.15.123:0/1682315446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c1612e0 0x7f039c161730 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f038c0099b0 tx=0x7f038c009250 comp rx=0 tx=0).stop 2023-10-06T04:31:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.967+0000 7f1deeb3d700 1 -- 172.21.15.123:0/2634475650 shutdown_connections 2023-10-06T04:31:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.967+0000 7f1deeb3d700 1 -- 172.21.15.123:0/2634475650 wait complete. 2023-10-06T04:31:42.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f039b59e700 1 -- 172.21.15.123:0/1682315446 shutdown_connections 2023-10-06T04:31:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f039b59e700 1 --2- 172.21.15.123:0/1682315446 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c06d860 0x7f039c06dc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f039b59e700 1 --2- 172.21.15.123:0/1682315446 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c1612e0 0x7f039c161730 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f039b59e700 1 -- 172.21.15.123:0/1682315446 >> 172.21.15.123:0/1682315446 conn(0x7f039c0cbd20 msgr2=0x7f039c0cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:42.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7effb9e2d700 1 Processor -- start 2023-10-06T04:31:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f5880ce4700 1 -- 172.21.15.123:0/311127642 shutdown_connections 2023-10-06T04:31:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f5880ce4700 1 --2- 172.21.15.123:0/311127642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c06e190 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f5880ce4700 1 --2- 172.21.15.123:0/311127642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c069ea0 0x7f587c0c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7f5880ce4700 1 -- 172.21.15.123:0/311127642 >> 172.21.15.123:0/311127642 conn(0x7f587c067660 msgr2=0x7f587c067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:42.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.968+0000 7effb9e2d700 1 -- start start 2023-10-06T04:31:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.969+0000 7f1deeb3d700 1 Processor -- start 2023-10-06T04:31:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.969+0000 7f1deeb3d700 1 -- start start 2023-10-06T04:31:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb9e2d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb4068d60 0x7effb41483e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb9e2d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb406a5d0 0x7effb4154840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb9e2d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb4153970 0x7effb40c25f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb9e2d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb4148920 con 0x7effb4068d60 2023-10-06T04:31:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb9e2d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effb413bb00 con 0x7effb406a5d0 2023-10-06T04:31:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb9e2d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effb4154410 con 0x7effb4153970 2023-10-06T04:31:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1deeb3d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de806f180 0x7f1de815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1deeb3d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de815d5c0 0x7f1de815b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.982 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1deeb3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de8161aa0 0x7f1de80c02e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1deeb3d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80c4ac0 con 0x7f1de8161aa0 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1deeb3d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de815b600 con 0x7f1de815d5c0 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1deeb3d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80c4190 con 0x7f1de806f180 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1dec8d9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de806f180 0x7f1de815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1dec8d9700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de806f180 0x7f1de815b820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46590/0 (socket says 172.21.15.123:46590) 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1dec8d9700 1 -- 172.21.15.123:0/4044840012 learned_addr learned my addr 172.21.15.123:0/4044840012 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1de7fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de815d5c0 0x7f1de815b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7f1ded0da700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de8161aa0 0x7f1de80c02e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effabfff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb406a5d0 0x7effb4154840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb4068d60 0x7effb41483e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb4068d60 0x7effb41483e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43412/0 (socket says 172.21.15.123:43412) 2023-10-06T04:31:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb37fe700 1 -- 172.21.15.123:0/2721728448 learned_addr learned my addr 172.21.15.123:0/2721728448 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb3fff700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb4153970 0x7effb40c25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7effb37fe700 1 -- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb4153970 msgr2=0x7effb40c25f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7effb37fe700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb4153970 0x7effb40c25f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7ff7eb7c7700 1 Processor -- start 2023-10-06T04:31:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.970+0000 7ff7eb7c7700 1 -- start start 2023-10-06T04:31:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7effb37fe700 1 -- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb406a5d0 msgr2=0x7effb4154840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7effb37fe700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb406a5d0 0x7effb4154840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7effb37fe700 1 -- 172.21.15.123:0/2721728448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c0094f0 con 0x7effb4068d60 2023-10-06T04:31:42.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7effb3fff700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb4153970 0x7effb40c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f1dec8d9700 1 -- 172.21.15.123:0/4044840012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de815d5c0 msgr2=0x7f1de815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f1dec8d9700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de815d5c0 0x7f1de815b0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f1dec8d9700 1 -- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de8161aa0 msgr2=0x7f1de80c02e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f1dec8d9700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de8161aa0 0x7f1de80c02e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f1dec8d9700 1 -- 172.21.15.123:0/4044840012 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd40094f0 con 0x7f1de806f180 2023-10-06T04:31:42.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f5880ce4700 1 -- 172.21.15.123:0/311127642 shutdown_connections 2023-10-06T04:31:42.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7effb37fe700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb4068d60 0x7effb41483e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7effa400c810 tx=0x7effa4007f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f5880ce4700 1 -- 172.21.15.123:0/311127642 wait complete. 2023-10-06T04:31:42.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f1dec8d9700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de806f180 0x7f1de815b820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1de806ce50 tx=0x7f1dd4016000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7ff7eb7c7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7e40680b0 0x7ff7e4068550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7ff7eb7c7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff7e40ce020 0x7ff7e40ce450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7ff7eb7c7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7e4098fd0 con 0x7ff7e415d870 2023-10-06T04:31:42.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7ff7eb7c7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e4068a90 con 0x7ff7e40ce020 2023-10-06T04:31:42.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7ff7eb7c7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7e40981b0 con 0x7ff7e40680b0 2023-10-06T04:31:42.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f039b59e700 1 -- 172.21.15.123:0/1682315446 shutdown_connections 2023-10-06T04:31:42.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7ff7e8d62700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7e40680b0 0x7ff7e4068550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7ff7e8d62700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7e40680b0 0x7ff7e4068550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41246/0 (socket says 172.21.15.123:41246) 2023-10-06T04:31:42.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7ff7e8d62700 1 -- 172.21.15.123:0/2013642016 learned_addr learned my addr 172.21.15.123:0/2013642016 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7effab7fe700 1 -- 172.21.15.123:0/2721728448 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4011590 con 0x7effb4068d60 2023-10-06T04:31:42.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7effab7fe700 1 -- 172.21.15.123:0/2721728448 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effa40116f0 con 0x7effb4068d60 2023-10-06T04:31:42.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7effab7fe700 1 -- 172.21.15.123:0/2721728448 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effa4011ab0 con 0x7effb4068d60 2023-10-06T04:31:42.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7ff7e8d62700 1 -- 172.21.15.123:0/2013642016 >> v2:172.21.15.123:3300/0 conn(0x7ff7e40ce020 msgr2=0x7ff7e40ce450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:42.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7ff7e8d62700 1 --2- 172.21.15.123:0/2013642016 >> v2:172.21.15.123:3300/0 conn(0x7ff7e40ce020 0x7ff7e40ce450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7ff7e8d62700 1 -- 172.21.15.123:0/2013642016 >> v1:172.21.15.177:6789/0 conn(0x7ff7e415d870 legacy=0x7ff7e40adf40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-10-06T04:31:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7ff7e8d62700 1 -- 172.21.15.123:0/2013642016 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7e40983c0 con 0x7ff7e40680b0 2023-10-06T04:31:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effb4153e50 con 0x7effb4068d60 2023-10-06T04:31:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.972+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effb40c38e0 con 0x7effb4068d60 2023-10-06T04:31:42.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.973+0000 7f5880ce4700 1 Processor -- start 2023-10-06T04:31:42.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.973+0000 7f1de4ff9700 1 -- 172.21.15.123:0/4044840012 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd4006bc0 con 0x7f1de806f180 2023-10-06T04:31:42.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.973+0000 7f1de4ff9700 1 -- 172.21.15.123:0/4044840012 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd4020070 con 0x7f1de806f180 2023-10-06T04:31:42.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.973+0000 7f1de4ff9700 1 -- 172.21.15.123:0/4044840012 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd401bcd0 con 0x7f1de806f180 2023-10-06T04:31:42.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.973+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de80c4470 con 0x7f1de806f180 2023-10-06T04:31:42.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.973+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de80c0820 con 0x7f1de806f180 2023-10-06T04:31:42.993 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.971+0000 7f039b59e700 1 -- 172.21.15.123:0/1682315446 wait complete. 2023-10-06T04:31:42.993 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.973+0000 7f039b59e700 1 Processor -- start 2023-10-06T04:31:42.993 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.974+0000 7ff7e8d62700 1 --2- 172.21.15.123:0/2013642016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e40680b0 0x7ff7e4068550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40767c0 tx=0x7ff7d4009250 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6a674a919d3d1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:42.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.974+0000 7effab7fe700 1 -- 172.21.15.123:0/2721728448 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7effa401b3b0 con 0x7effb4068d60 2023-10-06T04:31:42.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.974+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effb406c9b0 con 0x7effb4068d60 2023-10-06T04:31:42.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.974+0000 7f5880ce4700 1 -- start start 2023-10-06T04:31:42.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.975+0000 7f5880ce4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f587c069ea0 0x7f587c159200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.975+0000 7f5880ce4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c1570e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.975+0000 7f5880ce4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c15c880 0x7f587c157620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.975+0000 7f5880ce4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c140d10 con 0x7f587c069ea0 2023-10-06T04:31:42.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.975+0000 7f5880ce4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587c13fd00 con 0x7f587c0c8bc0 2023-10-06T04:31:42.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f5880ce4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f587c140880 con 0x7f587c15c880 2023-10-06T04:31:42.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.974+0000 7f039b59e700 1 -- start start 2023-10-06T04:31:42.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c06d860 0x7f039c15ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039b59e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c0c9f60 0x7f039c152960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039b59e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f039c1612e0 0x7f039c0c18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039b59e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1691e0 con 0x7f039c06d860 2023-10-06T04:31:42.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039b59e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1535b0 con 0x7f039c1612e0 2023-10-06T04:31:42.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039b59e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f039c1532d0 con 0x7f039c0c9f60 2023-10-06T04:31:42.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.975+0000 7effab7fe700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:42.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.975+0000 7effab7fe700 1 -- 172.21.15.123:0/2721728448 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7effa4083ca0 con 0x7effb4068d60 2023-10-06T04:31:42.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f5879d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c1570e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f5879d9b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c1570e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41248/0 (socket says 172.21.15.123:41248) 2023-10-06T04:31:42.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f5879d9b700 1 -- 172.21.15.123:0/1389564099 learned_addr learned my addr 172.21.15.123:0/1389564099 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:42.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f587ad9d700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c15c880 0x7f587c157620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f587a59c700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f587c069ea0 0x7f587c159200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f587ad9d700 1 -- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 msgr2=0x7f587c1570e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f587ad9d700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c1570e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f587ad9d700 1 -- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f587c069ea0 msgr2=0x7f587c159200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f587ad9d700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f587c069ea0 0x7f587c159200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f587ad9d700 1 -- 172.21.15.123:0/1389564099 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870009420 con 0x7f587c15c880 2023-10-06T04:31:42.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7ff7dbfff700 1 -- 172.21.15.123:0/2013642016 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400b820 con 0x7ff7e40680b0 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7ff7dbfff700 1 -- 172.21.15.123:0/2013642016 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7d400b980 con 0x7ff7e40680b0 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7ff7dbfff700 1 -- 172.21.15.123:0/2013642016 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7d400bd20 con 0x7ff7e40680b0 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/2013642016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e40680b0 msgr2=0x7ff7e4068550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/2013642016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e40680b0 0x7ff7e4068550 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40767c0 tx=0x7ff7d4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/2013642016 shutdown_connections 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/2013642016 >> v2:172.21.15.123:3300/0 conn(0x7ff7e40ce020 0x7ff7e40ce450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/2013642016 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e40680b0 0x7ff7e4068550 secure :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40767c0 tx=0x7ff7d4009250 comp rx=0 tx=0).stop 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/2013642016 >> 172.21.15.123:0/2013642016 conn(0x7ff7e4063230 msgr2=0x7ff7e4063630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:42.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/2013642016 shutdown_connections 2023-10-06T04:31:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f1de4ff9700 1 -- 172.21.15.123:0/4044840012 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1dd401c450 con 0x7f1de806f180 2023-10-06T04:31:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f1de4ff9700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc067080 0x7f1dcc069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f1de4ff9700 1 -- 172.21.15.123:0/4044840012 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1dd4022070 con 0x7f1de806f180 2023-10-06T04:31:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.978+0000 7f1de7fff700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc067080 0x7f1dcc069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.000 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.978+0000 7f587ad9d700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c15c880 0x7f587c157620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58700093f0 tx=0x7f5870027000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039ad9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f039c1612e0 0x7f039c0c18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039ad9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f039c1612e0 0x7f039c0c18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41250/0 (socket says 172.21.15.123:41250) 2023-10-06T04:31:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.976+0000 7f039ad9d700 1 -- 172.21.15.123:0/2833118855 learned_addr learned my addr 172.21.15.123:0/2833118855 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f039ad9d700 1 -- 172.21.15.123:0/2833118855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c0c9f60 msgr2=0x7f039c152960 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:31:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f039ad9d700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c0c9f60 0x7f039c152960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f039ad9d700 1 -- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c06d860 msgr2=0x7f039c15ac80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-10-06T04:31:43.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f039ad9d700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c06d860 0x7f039c15ac80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7f039ad9d700 1 -- 172.21.15.123:0/2833118855 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f038c0094f0 con 0x7f039c1612e0 2023-10-06T04:31:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.978+0000 7f039ad9d700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f039c1612e0 0x7f039c0c18f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f039400bbb0 tx=0x7f0394009840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.978+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1dc80052b0 con 0x7f1de806f180 2023-10-06T04:31:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.978+0000 7f1de7fff700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc067080 0x7f1dcc069530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1de815e850 tx=0x7f1ddc007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7f038a7fc700 1 -- 172.21.15.123:0/2833118855 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03940092e0 con 0x7f039c1612e0 2023-10-06T04:31:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.977+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/2013642016 wait complete. 2023-10-06T04:31:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7ff7eb7c7700 1 Processor -- start 2023-10-06T04:31:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7f038a7fc700 1 -- 172.21.15.123:0/2833118855 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f039400c7a0 con 0x7f039c1612e0 2023-10-06T04:31:43.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7f038a7fc700 1 -- 172.21.15.123:0/2833118855 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f039400cb60 con 0x7f039c1612e0 2023-10-06T04:31:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f039c168b90 con 0x7f039c1612e0 2023-10-06T04:31:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f039c15b1c0 con 0x7f039c1612e0 2023-10-06T04:31:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.978+0000 7effabfff700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7effabfff700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff94066f30 0x7eff940693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0099b0 tx=0x7eff9c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.980+0000 7f586a7fc700 1 -- 172.21.15.123:0/1389564099 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587000bbf0 con 0x7f587c15c880 2023-10-06T04:31:43.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.980+0000 7f586a7fc700 1 -- 172.21.15.123:0/1389564099 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f587000bd50 con 0x7f587c15c880 2023-10-06T04:31:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.980+0000 7f586a7fc700 1 -- 172.21.15.123:0/1389564099 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f587002b6a0 con 0x7f587c15c880 2023-10-06T04:31:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.979+0000 7ff7eb7c7700 1 -- start start 2023-10-06T04:31:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.980+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f587c13ff50 con 0x7f587c15c880 2023-10-06T04:31:43.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.980+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587c140510 con 0x7f587c15c880 2023-10-06T04:31:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.981+0000 7ff7eb7c7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7e40ce020 0x7ff7e4176860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.981+0000 7ff7eb7c7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e415d870 0x7ff7e415f610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.981+0000 7ff7eb7c7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e415fb50 0x7ff7e415e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.981+0000 7ff7eb7c7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e415dc40 con 0x7ff7e415d870 2023-10-06T04:31:43.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.981+0000 7ff7eb7c7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40c5e00 con 0x7ff7e415fb50 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.981+0000 7ff7eb7c7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7e40c60e0 con 0x7ff7e40ce020 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.982+0000 7ff7e9d64700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e415fb50 0x7ff7e415e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.982+0000 7ff7e9d64700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e415fb50 0x7ff7e415e690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41252/0 (socket says 172.21.15.123:41252) 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.982+0000 7ff7e9d64700 1 -- 172.21.15.123:0/1613843252 learned_addr learned my addr 172.21.15.123:0/1613843252 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.982+0000 7ff7e9563700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7e40ce020 0x7ff7e4176860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.982+0000 7ff7e8d62700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e415d870 0x7ff7e415f610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7ff7e9563700 1 -- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e415fb50 msgr2=0x7ff7e415e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7ff7e9563700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e415fb50 0x7ff7e415e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7ff7e9563700 1 -- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e415d870 msgr2=0x7ff7e415f610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7ff7e9563700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e415d870 0x7ff7e415f610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7ff7e9563700 1 -- 172.21.15.123:0/1613843252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7d40094f0 con 0x7ff7e40ce020 2023-10-06T04:31:43.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7f038a7fc700 1 -- 172.21.15.123:0/2833118855 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0394012070 con 0x7f039c1612e0 2023-10-06T04:31:43.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7effab7fe700 1 -- 172.21.15.123:0/2721728448 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7effa4050d90 con 0x7effb4068d60 2023-10-06T04:31:43.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7f1de4ff9700 1 -- 172.21.15.123:0/4044840012 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dd4053440 con 0x7f1de806f180 2023-10-06T04:31:43.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.984+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58600052b0 con 0x7f587c15c880 2023-10-06T04:31:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7ff7e9563700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7e40ce020 0x7ff7e4176860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000b510 tx=0x7ff7e0009060 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.983+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f039c1689f0 con 0x7f039c1612e0 2023-10-06T04:31:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.984+0000 7f038a7fc700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f037c067050 0x7f037c069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.984+0000 7f038a7fc700 1 -- 172.21.15.123:0/2833118855 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0394084410 con 0x7f039c1612e0 2023-10-06T04:31:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.985+0000 7f586a7fc700 1 -- 172.21.15.123:0/1389564099 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f587002bad0 con 0x7f587c15c880 2023-10-06T04:31:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.985+0000 7f586a7fc700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f585c067080 0x7f585c069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.986+0000 7f586a7fc700 1 -- 172.21.15.123:0/1389564099 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f58700973c0 con 0x7f587c15c880 2023-10-06T04:31:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.986+0000 7f587a59c700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f585c067080 0x7f585c069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.986+0000 7f587a59c700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f585c067080 0x7f585c069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f586c007fd0 tx=0x7f586c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.990+0000 7f586a7fc700 1 -- 172.21.15.123:0/1389564099 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f58700644b0 con 0x7f587c15c880 2023-10-06T04:31:43.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.989+0000 7ff7d97fa700 1 -- 172.21.15.123:0/1613843252 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e000be60 con 0x7ff7e40ce020 2023-10-06T04:31:43.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.989+0000 7ff7d97fa700 1 -- 172.21.15.123:0/1613843252 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7e000f510 con 0x7ff7e40ce020 2023-10-06T04:31:43.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.989+0000 7ff7d97fa700 1 -- 172.21.15.123:0/1613843252 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7e000f8b0 con 0x7ff7e40ce020 2023-10-06T04:31:43.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.989+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7e415de90 con 0x7ff7e40ce020 2023-10-06T04:31:43.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.989+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7e4176da0 con 0x7ff7e40ce020 2023-10-06T04:31:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.989+0000 7f038a7fc700 1 -- 172.21.15.123:0/2833118855 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f039c1689f0 con 0x7f039c1612e0 2023-10-06T04:31:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.993+0000 7ff7d97fa700 1 -- 172.21.15.123:0/1613843252 <== mon.2 v2:172.21.15.177:3301/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff7e000fc60 con 0x7ff7e40ce020 2023-10-06T04:31:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.993+0000 7ff7d97fa700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7c8067080 0x7ff7c8069530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.993+0000 7ff7d97fa700 1 -- 172.21.15.123:0/1613843252 <== mon.2 v2:172.21.15.177:3301/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff7e0083f10 con 0x7ff7e40ce020 2023-10-06T04:31:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.994+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7c40052b0 con 0x7ff7e40ce020 2023-10-06T04:31:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.995+0000 7f039a59c700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f037c067050 0x7f037c069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:42.996+0000 7f039a59c700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f037c067050 0x7f037c069500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0390009b10 tx=0x7f03900091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.013 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.001+0000 7ff7e8d62700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7c8067080 0x7ff7c8069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.001+0000 7ff7e8d62700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7c8067080 0x7ff7c8069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40767c0 tx=0x7ff7d4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.003+0000 7ff7d97fa700 1 -- 172.21.15.123:0/1613843252 <== mon.2 v2:172.21.15.177:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff7e0051080 con 0x7ff7e40ce020 2023-10-06T04:31:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.035+0000 7f351e363700 1 Processor -- start 2023-10-06T04:31:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.035+0000 7f351e363700 1 -- start start 2023-10-06T04:31:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f351e363700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35180c7260 0x7f35180c66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f351e363700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f35180877c0 0x7f3518069570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f351e363700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3518087c20 con 0x7f35181665a0 2023-10-06T04:31:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f351e363700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35180c6be0 con 0x7f35180c7260 2023-10-06T04:31:43.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f351e363700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3518166af0 con 0x7f35180877c0 2023-10-06T04:31:43.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f3517fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35180c7260 0x7f35180c66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f3517fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f35180c7260 0x7f35180c66a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43444/0 (socket says 172.21.15.123:43444) 2023-10-06T04:31:43.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.037+0000 7f3517fff700 1 -- 172.21.15.123:0/3740093175 learned_addr learned my addr 172.21.15.123:0/3740093175 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.038+0000 7f351c900700 1 --2- 172.21.15.123:0/3740093175 >> v2:172.21.15.177:3300/0 conn(0x7f35180877c0 0x7f3518069570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.040+0000 7f3517fff700 1 -- 172.21.15.123:0/3740093175 >> v2:172.21.15.177:3300/0 conn(0x7f35180877c0 msgr2=0x7f3518069570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.040+0000 7f3517fff700 1 --2- 172.21.15.123:0/3740093175 >> v2:172.21.15.177:3300/0 conn(0x7f35180877c0 0x7f3518069570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.040+0000 7f3517fff700 1 -- 172.21.15.123:0/3740093175 >> v1:172.21.15.123:6789/0 conn(0x7f35181665a0 legacy=0x7f35180870b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.040+0000 7f3517fff700 1 -- 172.21.15.123:0/3740093175 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3518166dd0 con 0x7f35180c7260 2023-10-06T04:31:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.041+0000 7f3517fff700 1 --2- 172.21.15.123:0/3740093175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180c7260 0x7f35180c66a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3518065a70 tx=0x7f3510007510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7b67f505dea7cf0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.039+0000 7f3516ffd700 1 -- 172.21.15.123:0/3740093175 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660571158 0 0) 0x7f3518087c20 con 0x7f35181665a0 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.041+0000 7f3516ffd700 1 -- 172.21.15.123:0/3740093175 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351000bc30 con 0x7f35180c7260 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.041+0000 7f3516ffd700 1 -- 172.21.15.123:0/3740093175 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351000bd90 con 0x7f35180c7260 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.041+0000 7f3516ffd700 1 -- 172.21.15.123:0/3740093175 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3510018aa0 con 0x7f35180c7260 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.042+0000 7f351e363700 1 -- 172.21.15.123:0/3740093175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180c7260 msgr2=0x7f35180c66a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.042+0000 7f351e363700 1 --2- 172.21.15.123:0/3740093175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180c7260 0x7f35180c66a0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f3518065a70 tx=0x7f3510007510 comp rx=0 tx=0).stop 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.043+0000 7f351e363700 1 -- 172.21.15.123:0/3740093175 shutdown_connections 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.043+0000 7f351e363700 1 --2- 172.21.15.123:0/3740093175 >> v2:172.21.15.177:3300/0 conn(0x7f35180877c0 0x7f3518069570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.043+0000 7f351e363700 1 --2- 172.21.15.123:0/3740093175 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180c7260 0x7f35180c66a0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.043+0000 7f351e363700 1 -- 172.21.15.123:0/3740093175 >> 172.21.15.123:0/3740093175 conn(0x7f3518067660 msgr2=0x7f3518067a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.043+0000 7f351e363700 1 -- 172.21.15.123:0/3740093175 shutdown_connections 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.043+0000 7f351e363700 1 -- 172.21.15.123:0/3740093175 wait complete. 2023-10-06T04:31:43.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.045+0000 7f351e363700 1 Processor -- start 2023-10-06T04:31:43.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.045+0000 7f351e363700 1 -- start start 2023-10-06T04:31:43.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f351e363700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180877c0 0x7f3518179af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f351e363700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35180c7260 0x7f35180adc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f351e363700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35181665a0 0x7f35180bf950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f351e363700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f351817a030 con 0x7f35180877c0 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f351e363700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3518176790 con 0x7f35181665a0 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f351e363700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3518176490 con 0x7f35180c7260 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f3517fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180877c0 0x7f3518179af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f3517fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180877c0 0x7f3518179af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43450/0 (socket says 172.21.15.123:43450) 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f3517fff700 1 -- 172.21.15.123:0/1005142453 learned_addr learned my addr 172.21.15.123:0/1005142453 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f351c900700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35181665a0 0x7f35180bf950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.047+0000 7f35177fe700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35180c7260 0x7f35180adc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.048+0000 7f351c900700 1 -- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35180c7260 msgr2=0x7f35180adc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.048+0000 7f351c900700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35180c7260 0x7f35180adc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.048+0000 7f351c900700 1 -- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180877c0 msgr2=0x7f3518179af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.048+0000 7f351c900700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180877c0 0x7f3518179af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.048+0000 7f351c900700 1 -- 172.21.15.123:0/1005142453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3510009ae0 con 0x7f35181665a0 2023-10-06T04:31:43.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.048+0000 7f351c900700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35181665a0 0x7f35180bf950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3508007810 tx=0x7f350800e040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.050+0000 7f3507fff700 1 -- 172.21.15.123:0/1005142453 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f350801a070 con 0x7f35181665a0 2023-10-06T04:31:43.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.050+0000 7f3507fff700 1 -- 172.21.15.123:0/1005142453 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3508009740 con 0x7f35181665a0 2023-10-06T04:31:43.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.050+0000 7f3507fff700 1 -- 172.21.15.123:0/1005142453 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3508009ae0 con 0x7f35181665a0 2023-10-06T04:31:43.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.050+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f351817a260 con 0x7f35181665a0 2023-10-06T04:31:43.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.050+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f351817a820 con 0x7f35181665a0 2023-10-06T04:31:43.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.054+0000 7f3507fff700 1 -- 172.21.15.123:0/1005142453 <== mon.1 v2:172.21.15.177:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f350801f410 con 0x7f35181665a0 2023-10-06T04:31:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.054+0000 7f3507fff700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34fc067050 0x7f34fc069500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.054+0000 7f3507fff700 1 -- 172.21.15.123:0/1005142453 <== mon.1 v2:172.21.15.177:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3508011070 con 0x7f35181665a0 2023-10-06T04:31:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.055+0000 7f3517fff700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34fc067050 0x7f34fc069500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:43.057 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.055+0000 7f3517fff700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34fc067050 0x7f34fc069500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3510007550 tx=0x7f351001e000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:43.058 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.056+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34f40052b0 con 0x7f35181665a0 2023-10-06T04:31:43.065 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.063+0000 7f3507fff700 1 -- 172.21.15.123:0/1005142453 <== mon.1 v2:172.21.15.177:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3508051ba0 con 0x7f35181665a0 2023-10-06T04:31:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.269+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7fd28c005560 con 0x7fd2a8155530 2023-10-06T04:31:43.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.270+0000 7fd2967fc700 1 -- 172.21.15.123:0/3876135962 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd2a0018030 con 0x7fd2a8155530 2023-10-06T04:31:43.275 INFO:teuthology.orchestra.run.smithi123.stdout:55834574854 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd288067080 msgr2=0x7fd288069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd288067080 0x7fd288069530 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd29800bc90 tx=0x7fd298008040 comp rx=0 tx=0).stop 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a8155530 msgr2=0x7fd2a8151270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a8155530 0x7fd2a8151270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fd2a0001bb0 tx=0x7fd2a0006180 comp rx=0 tx=0).stop 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 shutdown_connections 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd288067080 0x7fd288069530 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2a8155530 0x7fd2a8151270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2a806dac0 0x7fd2a8152b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 --2- 172.21.15.123:0/3876135962 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd2a8155160 0x7fd2a8150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.273+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 >> 172.21.15.123:0/3876135962 conn(0x7fd2a8067660 msgr2=0x7fd2a80cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.278+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 shutdown_connections 2023-10-06T04:31:43.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.278+0000 7fd2acdae700 1 -- 172.21.15.123:0/3876135962 wait complete. 2023-10-06T04:31:43.298 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574854 for osd.2 2023-10-06T04:31:43.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.303+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7fb074005560 con 0x7fb08c0cca10 2023-10-06T04:31:43.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.311+0000 7fb0827fc700 1 -- 172.21.15.123:0/630831131 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb084054c70 con 0x7fb08c0cca10 2023-10-06T04:31:43.313 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb070067080 msgr2=0x7fb070069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb070067080 0x7fb070069530 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fb08800a010 tx=0x7fb088017000 comp rx=0 tx=0).stop 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 msgr2=0x7fb08c143a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c143a30 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fb08400c940 tx=0x7fb0840016d0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 shutdown_connections 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb070067080 0x7fb070069530 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb08c0bf330 0x7fb08c0b0c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb08c0cca10 0x7fb08c143a30 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 --2- 172.21.15.123:0/630831131 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb08c0c3390 0x7fb08c14df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 >> 172.21.15.123:0/630831131 conn(0x7fb08c0680f0 msgr2=0x7fb08c0c9a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 shutdown_connections 2023-10-06T04:31:43.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.314+0000 7fb0943db700 1 -- 172.21.15.123:0/630831131 wait complete. 2023-10-06T04:31:43.334 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.3 2023-10-06T04:31:43.334 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:43.339 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.336+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f1dc8005560 con 0x7f1de806f180 2023-10-06T04:31:43.339 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.337+0000 7f1de4ff9700 1 -- 172.21.15.123:0/4044840012 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f1dd4056a60 con 0x7f1de806f180 2023-10-06T04:31:43.339 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc067080 msgr2=0x7f1dcc069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc067080 0x7f1dcc069530 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1de815e850 tx=0x7f1ddc007ea0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de806f180 msgr2=0x7f1de815b820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de806f180 0x7f1de815b820 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f1de806ce50 tx=0x7f1dd4016000 comp rx=0 tx=0).stop 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 shutdown_connections 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc067080 0x7f1dcc069530 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1de8161aa0 0x7f1de80c02e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.342 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de806f180 0x7f1de815b820 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 --2- 172.21.15.123:0/4044840012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de815d5c0 0x7f1de815b0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.339+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 >> 172.21.15.123:0/4044840012 conn(0x7f1de8067660 msgr2=0x7f1de80c7e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.340+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 shutdown_connections 2023-10-06T04:31:43.343 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.340+0000 7f1deeb3d700 1 -- 172.21.15.123:0/4044840012 wait complete. 2023-10-06T04:31:43.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.348+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f039c1689f0 con 0x7f039c1612e0 2023-10-06T04:31:43.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.349+0000 7f038a7fc700 1 -- 172.21.15.123:0/2833118855 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f039c1689f0 con 0x7f039c1612e0 2023-10-06T04:31:43.351 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:43.353 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.351+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f037c067050 msgr2=0x7f037c069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.351+0000 7f039b59e700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f037c067050 0x7f037c069500 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f0390009b10 tx=0x7f03900091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.351+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f039c1612e0 msgr2=0x7f039c0c18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.351+0000 7f039b59e700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f039c1612e0 0x7f039c0c18f0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f039400bbb0 tx=0x7f0394009840 comp rx=0 tx=0).stop 2023-10-06T04:31:43.355 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.0 2023-10-06T04:31:43.355 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:43.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.352+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 shutdown_connections 2023-10-06T04:31:43.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.352+0000 7f039b59e700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f037c067050 0x7f037c069500 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.352+0000 7f039b59e700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f039c06d860 0x7f039c15ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.352+0000 7f039b59e700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f039c1612e0 0x7f039c0c18f0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.352+0000 7f039b59e700 1 --2- 172.21.15.123:0/2833118855 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f039c0c9f60 0x7f039c152960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.352+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 >> 172.21.15.123:0/2833118855 conn(0x7f039c0cbd20 msgr2=0x7f039c15bce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.357+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 shutdown_connections 2023-10-06T04:31:43.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.358+0000 7f039b59e700 1 -- 172.21.15.123:0/2833118855 wait complete. 2023-10-06T04:31:43.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.368+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7effb40c7350 con 0x7effb4068d60 2023-10-06T04:31:43.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.368+0000 7effab7fe700 1 -- 172.21.15.123:0/2721728448 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7effa4017020 con 0x7effb4068d60 2023-10-06T04:31:43.371 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:43.372 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.5 2023-10-06T04:31:43.372 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:43.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.371+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff94066f30 msgr2=0x7eff940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.371+0000 7effb9e2d700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff94066f30 0x7eff940693e0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7eff9c0099b0 tx=0x7eff9c016000 comp rx=0 tx=0).stop 2023-10-06T04:31:43.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.371+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb4068d60 msgr2=0x7effb41483e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.371+0000 7effb9e2d700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb4068d60 0x7effb41483e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7effa400c810 tx=0x7effa4007f30 comp rx=0 tx=0).stop 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 shutdown_connections 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff94066f30 0x7eff940693e0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effb4068d60 0x7effb41483e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effb4153970 0x7effb40c25f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 --2- 172.21.15.123:0/2721728448 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effb406a5d0 0x7effb4154840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 >> 172.21.15.123:0/2721728448 conn(0x7effb40c9d00 msgr2=0x7effb40ca340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 shutdown_connections 2023-10-06T04:31:43.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.372+0000 7effb9e2d700 1 -- 172.21.15.123:0/2721728448 wait complete. 2023-10-06T04:31:43.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.378+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f5860005560 con 0x7f587c15c880 2023-10-06T04:31:43.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.378+0000 7f586a7fc700 1 -- 172.21.15.123:0/1389564099 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5870067ad0 con 0x7f587c15c880 2023-10-06T04:31:43.381 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:43.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.381+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f585c067080 msgr2=0x7f585c069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.381+0000 7f5880ce4700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f585c067080 0x7f585c069530 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f586c007fd0 tx=0x7f586c009040 comp rx=0 tx=0).stop 2023-10-06T04:31:43.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.381+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c15c880 msgr2=0x7f587c157620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.381+0000 7f5880ce4700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c15c880 0x7f587c157620 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f58700093f0 tx=0x7f5870027000 comp rx=0 tx=0).stop 2023-10-06T04:31:43.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 shutdown_connections 2023-10-06T04:31:43.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f585c067080 0x7f585c069530 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f587c069ea0 0x7f587c159200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f587c15c880 0x7f587c157620 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 --2- 172.21.15.123:0/1389564099 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f587c0c8bc0 0x7f587c1570e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 >> 172.21.15.123:0/1389564099 conn(0x7f587c067660 msgr2=0x7f587c159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 shutdown_connections 2023-10-06T04:31:43.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.383+0000 7f5880ce4700 1 -- 172.21.15.123:0/1389564099 wait complete. 2023-10-06T04:31:43.390 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.1 2023-10-06T04:31:43.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.387+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7ff7c4005560 con 0x7ff7e40ce020 2023-10-06T04:31:43.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.387+0000 7ff7d97fa700 1 -- 172.21.15.123:0/1613843252 <== mon.2 v2:172.21.15.177:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff7e00546a0 con 0x7ff7e40ce020 2023-10-06T04:31:43.391 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:43.391 INFO:teuthology.orchestra.run.smithi123.stdout:55834574855 2023-10-06T04:31:43.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7c8067080 msgr2=0x7ff7c8069530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7c8067080 0x7ff7c8069530 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7ff7e40767c0 tx=0x7ff7d4016000 comp rx=0 tx=0).stop 2023-10-06T04:31:43.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7e40ce020 msgr2=0x7ff7e4176860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7e40ce020 0x7ff7e4176860 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff7e000b510 tx=0x7ff7e0009060 comp rx=0 tx=0).stop 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 shutdown_connections 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff7c8067080 0x7ff7c8069530 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7e415fb50 0x7ff7e415e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff7e40ce020 0x7ff7e4176860 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 --2- 172.21.15.123:0/1613843252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7e415d870 0x7ff7e415f610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 >> 172.21.15.123:0/1613843252 conn(0x7ff7e4063230 msgr2=0x7ff7e41576d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 shutdown_connections 2023-10-06T04:31:43.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.390+0000 7ff7eb7c7700 1 -- 172.21.15.123:0/1613843252 wait complete. 2023-10-06T04:31:43.398 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.7 2023-10-06T04:31:43.398 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:43.405 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574855 for osd.6 2023-10-06T04:31:43.405 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:43.420 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.418+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7f34f4005560 con 0x7f35181665a0 2023-10-06T04:31:43.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.418+0000 7f3507fff700 1 -- 172.21.15.123:0/1005142453 <== mon.1 v2:172.21.15.177:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f35080551c0 con 0x7f35181665a0 2023-10-06T04:31:43.421 INFO:teuthology.orchestra.run.smithi123.stdout:55834574857 2023-10-06T04:31:43.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34fc067050 msgr2=0x7f34fc069500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34fc067050 0x7f34fc069500 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3510007550 tx=0x7f351001e000 comp rx=0 tx=0).stop 2023-10-06T04:31:43.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35181665a0 msgr2=0x7f35180bf950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:43.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35181665a0 0x7f35180bf950 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f3508007810 tx=0x7f350800e040 comp rx=0 tx=0).stop 2023-10-06T04:31:43.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 shutdown_connections 2023-10-06T04:31:43.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34fc067050 0x7f34fc069500 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f35180877c0 0x7f3518179af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f35181665a0 0x7f35180bf950 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 --2- 172.21.15.123:0/1005142453 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f35180c7260 0x7f35180adc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 >> 172.21.15.123:0/1005142453 conn(0x7f3518067660 msgr2=0x7f35180c7c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 shutdown_connections 2023-10-06T04:31:43.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:43.420+0000 7f351e363700 1 -- 172.21.15.123:0/1005142453 wait complete. 2023-10-06T04:31:43.434 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574857 got 55834574857 for osd.4 2023-10-06T04:31:43.435 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:44.299 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-10-06T04:31:44.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.431+0000 7f79b6200700 1 Processor -- start 2023-10-06T04:31:44.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.431+0000 7f79b6200700 1 -- start start 2023-10-06T04:31:44.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79b6200700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79b006e960 0x7f79b0164330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79b6200700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b0068270 con 0x7f79b0142900 2023-10-06T04:31:44.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79b6200700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b01597f0 con 0x7f79b015db30 2023-10-06T04:31:44.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79b6200700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00c7cc0 con 0x7f79b006e960 2023-10-06T04:31:44.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79affff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f79b006e960 0x7f79b0164330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79af7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f79b0142900 0x7f79b00c75b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57012/0 (socket says 172.21.15.123:57012) 2023-10-06T04:31:44.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79af7fe700 1 -- 172.21.15.123:0/3017646354 learned_addr learned my addr 172.21.15.123:0/3017646354 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79affff700 1 -- 172.21.15.123:0/3017646354 >> v1:172.21.15.177:6789/0 conn(0x7f79b015db30 legacy=0x7f79b006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79affff700 1 -- 172.21.15.123:0/3017646354 >> v1:172.21.15.123:6789/0 conn(0x7f79b0142900 legacy=0x7f79b00c75b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79affff700 1 -- 172.21.15.123:0/3017646354 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b0067660 con 0x7f79b006e960 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.432+0000 7f79ae7fc700 1 -- 172.21.15.123:0/3017646354 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3796946006 0 0) 0x7f79b0068270 con 0x7f79b0142900 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79affff700 1 --2- 172.21.15.123:0/3017646354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b0164330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f79a80098e0 tx=0x7f79a80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93ed5c0538b5cd5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79ae7fc700 1 -- 172.21.15.123:0/3017646354 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8009e80 con 0x7f79b006e960 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79ae7fc700 1 -- 172.21.15.123:0/3017646354 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800b860 con 0x7f79b006e960 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79ae7fc700 1 -- 172.21.15.123:0/3017646354 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a800bbe0 con 0x7f79b006e960 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79b6200700 1 -- 172.21.15.123:0/3017646354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 msgr2=0x7f79b0164330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79b6200700 1 --2- 172.21.15.123:0/3017646354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b0164330 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f79a80098e0 tx=0x7f79a80091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79b6200700 1 -- 172.21.15.123:0/3017646354 shutdown_connections 2023-10-06T04:31:44.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79b6200700 1 --2- 172.21.15.123:0/3017646354 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b0164330 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79b6200700 1 -- 172.21.15.123:0/3017646354 >> 172.21.15.123:0/3017646354 conn(0x7f79b00cd010 msgr2=0x7f79b00cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:44.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79b6200700 1 -- 172.21.15.123:0/3017646354 shutdown_connections 2023-10-06T04:31:44.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.433+0000 7f79b6200700 1 -- 172.21.15.123:0/3017646354 wait complete. 2023-10-06T04:31:44.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 Processor -- start 2023-10-06T04:31:44.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 -- start start 2023-10-06T04:31:44.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b00c0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79b0142900 0x7f79b015ab60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79b015db30 0x7f79b015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b016a730 con 0x7f79b006e960 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015b740 con 0x7f79b015db30 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.435+0000 7f79b6200700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b015ba20 con 0x7f79b0142900 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b00c0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b00c0b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43472/0 (socket says 172.21.15.123:43472) 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 -- 172.21.15.123:0/538497555 learned_addr learned my addr 172.21.15.123:0/538497555 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79affff700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79b015db30 0x7f79b015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79aeffd700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79b0142900 0x7f79b015ab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 -- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79b0142900 msgr2=0x7f79b015ab60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79b0142900 0x7f79b015ab60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 -- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79b015db30 msgr2=0x7f79b015b0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79b015db30 0x7f79b015b0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 -- 172.21.15.123:0/538497555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a8009420 con 0x7f79b006e960 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79affff700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79b015db30 0x7f79b015b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79af7fe700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b00c0b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79a000ccc0 tx=0x7f79a0001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79a77fe700 1 -- 172.21.15.123:0/538497555 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a000ab70 con 0x7f79b006e960 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79a77fe700 1 -- 172.21.15.123:0/538497555 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a000acd0 con 0x7f79b006e960 2023-10-06T04:31:44.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b015bd00 con 0x7f79b006e960 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79a77fe700 1 -- 172.21.15.123:0/538497555 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a0007340 con 0x7f79b006e960 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.436+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b015c2c0 con 0x7f79b006e960 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.437+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b006d2b0 con 0x7f79b006e960 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.437+0000 7f79a77fe700 1 -- 172.21.15.123:0/538497555 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f79a0007770 con 0x7f79b006e960 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.438+0000 7f79a77fe700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7990066f30 0x7f79900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.438+0000 7f79a77fe700 1 -- 172.21.15.123:0/538497555 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f79a0084ab0 con 0x7f79b006e960 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.438+0000 7f79aeffd700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7990066f30 0x7f79900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.438+0000 7f79aeffd700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7990066f30 0x7f79900693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79980037f0 tx=0x7f7998009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:44.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.442+0000 7f79a77fe700 1 -- 172.21.15.123:0/538497555 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f79a0051c40 con 0x7f79b006e960 2023-10-06T04:31:44.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.624+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f79b0067940 con 0x7f79b006e960 2023-10-06T04:31:44.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.624+0000 7f79a77fe700 1 -- 172.21.15.123:0/538497555 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f79a0055260 con 0x7f79b006e960 2023-10-06T04:31:44.627 INFO:teuthology.orchestra.run.smithi123.stdout:55834574856 2023-10-06T04:31:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7990066f30 msgr2=0x7f79900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7990066f30 0x7f79900693e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f79980037f0 tx=0x7f7998009040 comp rx=0 tx=0).stop 2023-10-06T04:31:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 msgr2=0x7f79b00c0b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b00c0b70 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f79a000ccc0 tx=0x7f79a0001a80 comp rx=0 tx=0).stop 2023-10-06T04:31:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 shutdown_connections 2023-10-06T04:31:44.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7990066f30 0x7f79900693e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79b006e960 0x7f79b00c0b70 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f79b015db30 0x7f79b015b0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 --2- 172.21.15.123:0/538497555 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79b0142900 0x7f79b015ab60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 >> 172.21.15.123:0/538497555 conn(0x7f79b00cd010 msgr2=0x7f79b00cbf70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 shutdown_connections 2023-10-06T04:31:44.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.626+0000 7f79b6200700 1 -- 172.21.15.123:0/538497555 wait complete. 2023-10-06T04:31:44.640 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574856 got 55834574856 for osd.2 2023-10-06T04:31:44.640 DEBUG:teuthology.parallel:result is None 2023-10-06T04:31:44.641 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T04:31:44.641 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:31:44.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.756+0000 7f0e21eff700 1 Processor -- start 2023-10-06T04:31:44.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.756+0000 7f0e21eff700 1 -- start start 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e21eff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0e1c159410 0x7f0e1c0af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e21eff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c068d80 0x7f0e1c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e21eff700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e1c06af00 con 0x7f0e1c1597e0 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e21eff700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e1c0c8050 con 0x7f0e1c159410 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e21eff700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c06b460 con 0x7f0e1c068d80 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c068d80 0x7f0e1c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0e1c159410 0x7f0e1c0af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1b7fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f0e1c159410 0x7f0e1c0af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41288/0 (socket says 172.21.15.123:41288) 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1bfff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c068d80 0x7f0e1c0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46644/0 (socket says 172.21.15.123:46644) 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1affd700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f0e1c1597e0 0x7f0e1c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:53430/0 (socket says 172.21.15.123:53430) 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1b7fe700 1 -- 172.21.15.123:0/592622914 learned_addr learned my addr 172.21.15.123:0/592622914 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1bfff700 1 -- 172.21.15.123:0/592622914 >> v2:172.21.15.177:3300/0 conn(0x7f0e1c159410 msgr2=0x7f0e1c0af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1bfff700 1 --2- 172.21.15.123:0/592622914 >> v2:172.21.15.177:3300/0 conn(0x7f0e1c159410 0x7f0e1c0af6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1bfff700 1 -- 172.21.15.123:0/592622914 >> v1:172.21.15.177:6789/0 conn(0x7f0e1c1597e0 legacy=0x7f0e1c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1bfff700 1 -- 172.21.15.123:0/592622914 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e1c06b130 con 0x7f0e1c068d80 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1b7fe700 1 --2- 172.21.15.123:0/592622914 >> v2:172.21.15.177:3300/0 conn(0x7f0e1c159410 0x7f0e1c0af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.757+0000 7f0e1a7fc700 1 -- 172.21.15.123:0/592622914 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2425527539 0 0) 0x7f0e1c06af00 con 0x7f0e1c1597e0 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e1bfff700 1 --2- 172.21.15.123:0/592622914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c068d80 0x7f0e1c0c6dd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400bb90 tx=0x7f0e1400b430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b4694e0e0964804b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:44.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e1a7fc700 1 -- 172.21.15.123:0/592622914 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1400db40 con 0x7f0e1c068d80 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e1a7fc700 1 -- 172.21.15.123:0/592622914 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1400dca0 con 0x7f0e1c068d80 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e1a7fc700 1 -- 172.21.15.123:0/592622914 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140196b0 con 0x7f0e1c068d80 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 -- 172.21.15.123:0/592622914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c068d80 msgr2=0x7f0e1c0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 --2- 172.21.15.123:0/592622914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c068d80 0x7f0e1c0c6dd0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400bb90 tx=0x7f0e1400b430 comp rx=0 tx=0).stop 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 -- 172.21.15.123:0/592622914 shutdown_connections 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 --2- 172.21.15.123:0/592622914 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c068d80 0x7f0e1c0c6dd0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 --2- 172.21.15.123:0/592622914 >> v2:172.21.15.177:3300/0 conn(0x7f0e1c159410 0x7f0e1c0af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 -- 172.21.15.123:0/592622914 >> 172.21.15.123:0/592622914 conn(0x7f0e1c0c9c90 msgr2=0x7f0e1c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 -- 172.21.15.123:0/592622914 shutdown_connections 2023-10-06T04:31:44.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.758+0000 7f0e21eff700 1 -- 172.21.15.123:0/592622914 wait complete. 2023-10-06T04:31:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.759+0000 7f0e21eff700 1 Processor -- start 2023-10-06T04:31:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e21eff700 1 -- start start 2023-10-06T04:31:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e21eff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e1c068d80 0x7f0e1c0a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e21eff700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c159410 0x7f0e1c0abf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e21eff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1c1597e0 0x7f0e1c0c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e21eff700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0c6e10 con 0x7f0e1c1597e0 2023-10-06T04:31:44.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e21eff700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0a9de0 con 0x7f0e1c068d80 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e21eff700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e1c0ac4d0 con 0x7f0e1c159410 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1c1597e0 0x7f0e1c0c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e1bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1c1597e0 0x7f0e1c0c4630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43476/0 (socket says 172.21.15.123:43476) 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e1bfff700 1 -- 172.21.15.123:0/399780507 learned_addr learned my addr 172.21.15.123:0/399780507 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e1b7fe700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e1c068d80 0x7f0e1c0a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.760+0000 7f0e1affd700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c159410 0x7f0e1c0abf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e1bfff700 1 -- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c159410 msgr2=0x7f0e1c0abf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e1bfff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c159410 0x7f0e1c0abf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e1bfff700 1 -- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e1c068d80 msgr2=0x7f0e1c0a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e1bfff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e1c068d80 0x7f0e1c0a98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e1bfff700 1 -- 172.21.15.123:0/399780507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e0c009ae0 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e1bfff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1c1597e0 0x7f0e1c0c4630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400bfd0 tx=0x7f0e1401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e137fe700 1 -- 172.21.15.123:0/399780507 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140105c0 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e137fe700 1 -- 172.21.15.123:0/399780507 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e14010740 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1400b6d0 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e137fe700 1 -- 172.21.15.123:0/399780507 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e14010b00 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.761+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e1c0c76f0 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.762+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e1c0c5340 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.762+0000 7f0e137fe700 1 -- 172.21.15.123:0/399780507 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e1402d440 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.763+0000 7f0e137fe700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dfc066f30 0x7f0dfc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.763+0000 7f0e137fe700 1 -- 172.21.15.123:0/399780507 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f0e14020070 con 0x7f0e1c1597e0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.763+0000 7f0e1b7fe700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dfc066f30 0x7f0dfc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:44.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.763+0000 7f0e1b7fe700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dfc066f30 0x7f0dfc0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0072d0 tx=0x7f0e0c00b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:44.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.767+0000 7f0e137fe700 1 -- 172.21.15.123:0/399780507 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e14053870 con 0x7f0e1c1597e0 2023-10-06T04:31:44.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.945+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e1c04bb80 con 0x7f0dfc066f30 2023-10-06T04:31:44.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.952+0000 7f0e137fe700 1 -- 172.21.15.123:0/399780507 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286527 (secure 0 0 0) 0x7f0e1c04bb80 con 0x7f0dfc066f30 2023-10-06T04:31:44.961 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.955+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dfc066f30 msgr2=0x7f0dfc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.955+0000 7f0e21eff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dfc066f30 0x7f0dfc0693e0 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f0e0c0072d0 tx=0x7f0e0c00b040 comp rx=0 tx=0).stop 2023-10-06T04:31:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.955+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1c1597e0 msgr2=0x7f0e1c0c4630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.955+0000 7f0e21eff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1c1597e0 0x7f0e1c0c4630 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f0e1400bfd0 tx=0x7f0e1401b000 comp rx=0 tx=0).stop 2023-10-06T04:31:44.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 shutdown_connections 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0dfc066f30 0x7f0dfc0693e0 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e1c1597e0 0x7f0e1c0c4630 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e1c068d80 0x7f0e1c0a98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 --2- 172.21.15.123:0/399780507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e1c159410 0x7f0e1c0abf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 >> 172.21.15.123:0/399780507 conn(0x7f0e1c0c9c90 msgr2=0x7f0e1c0c83d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 shutdown_connections 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:44.956+0000 7f0e21eff700 1 -- 172.21.15.123:0/399780507 wait complete. 2023-10-06T04:31:44.966 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:31:44.970 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-06T04:31:44.891123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.853489"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066588+0000","last_change":"2023-10-06T04:31:35.057310+0000","last_active":"2023-10-06T04:31:35.066588+0000","last_peered":"2023-10-06T04:31:35.066588+0000","last_clean":"2023-10-06T04:31:35.066588+0000","last_became_active":"2023-10-06T04:31:35.057042+0000","last_became_peered":"2023-10-06T04:31:35.057042+0000","last_unstale":"2023-10-06T04:31:35.066588+0000","last_undegraded":"2023-10-06T04:31:35.066588+0000","last_fullsized":"2023-10-06T04:31:35.066588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065545+0000","last_change":"2023-10-06T04:31:35.063658+0000","last_active":"2023-10-06T04:31:35.065545+0000","last_peered":"2023-10-06T04:31:35.065545+0000","last_clean":"2023-10-06T04:31:35.065545+0000","last_became_active":"2023-10-06T04:31:35.063403+0000","last_became_peered":"2023-10-06T04:31:35.063403+0000","last_unstale":"2023-10-06T04:31:35.065545+0000","last_undegraded":"2023-10-06T04:31:35.065545+0000","last_fullsized":"2023-10-06T04:31:35.065545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059298+0000","last_change":"2023-10-06T04:31:35.056211+0000","last_active":"2023-10-06T04:31:35.059298+0000","last_peered":"2023-10-06T04:31:35.059298+0000","last_clean":"2023-10-06T04:31:35.059298+0000","last_became_active":"2023-10-06T04:31:35.055883+0000","last_became_peered":"2023-10-06T04:31:35.055883+0000","last_unstale":"2023-10-06T04:31:35.059298+0000","last_undegraded":"2023-10-06T04:31:35.059298+0000","last_fullsized":"2023-10-06T04:31:35.059298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:54.893111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060097+0000","last_change":"2023-10-06T04:31:35.055601+0000","last_active":"2023-10-06T04:31:35.060097+0000","last_peered":"2023-10-06T04:31:35.060097+0000","last_clean":"2023-10-06T04:31:35.060097+0000","last_became_active":"2023-10-06T04:31:35.055325+0000","last_became_peered":"2023-10-06T04:31:35.055325+0000","last_unstale":"2023-10-06T04:31:35.060097+0000","last_undegraded":"2023-10-06T04:31:35.060097+0000","last_fullsized":"2023-10-06T04:31:35.060097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063587+0000","last_change":"2023-10-06T04:31:35.061888+0000","last_active":"2023-10-06T04:31:35.063587+0000","last_peered":"2023-10-06T04:31:35.063587+0000","last_clean":"2023-10-06T04:31:35.063587+0000","last_became_active":"2023-10-06T04:31:35.061635+0000","last_became_peered":"2023-10-06T04:31:35.061635+0000","last_unstale":"2023-10-06T04:31:35.063587+0000","last_undegraded":"2023-10-06T04:31:35.063587+0000","last_fullsized":"2023-10-06T04:31:35.063587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059324+0000","last_change":"2023-10-06T04:31:35.055786+0000","last_active":"2023-10-06T04:31:35.059324+0000","last_peered":"2023-10-06T04:31:35.059324+0000","last_clean":"2023-10-06T04:31:35.059324+0000","last_became_active":"2023-10-06T04:31:35.055501+0000","last_became_peered":"2023-10-06T04:31:35.055501+0000","last_unstale":"2023-10-06T04:31:35.059324+0000","last_undegraded":"2023-10-06T04:31:35.059324+0000","last_fullsized":"2023-10-06T04:31:35.059324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:47.521895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066329+0000","last_change":"2023-10-06T04:31:35.065611+0000","last_active":"2023-10-06T04:31:35.066329+0000","last_peered":"2023-10-06T04:31:35.066329+0000","last_clean":"2023-10-06T04:31:35.066329+0000","last_became_active":"2023-10-06T04:31:35.064864+0000","last_became_peered":"2023-10-06T04:31:35.064864+0000","last_unstale":"2023-10-06T04:31:35.066329+0000","last_undegraded":"2023-10-06T04:31:35.066329+0000","last_fullsized":"2023-10-06T04:31:35.066329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064924+0000","last_change":"2023-10-06T04:31:35.060662+0000","last_active":"2023-10-06T04:31:35.064924+0000","last_peered":"2023-10-06T04:31:35.064924+0000","last_clean":"2023-10-06T04:31:35.064924+0000","last_became_active":"2023-10-06T04:31:35.060371+0000","last_became_peered":"2023-10-06T04:31:35.060371+0000","last_unstale":"2023-10-06T04:31:35.064924+0000","last_undegraded":"2023-10-06T04:31:35.064924+0000","last_fullsized":"2023-10-06T04:31:35.064924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063678+0000","last_change":"2023-10-06T04:31:35.061068+0000","last_active":"2023-10-06T04:31:35.063678+0000","last_peered":"2023-10-06T04:31:35.063678+0000","last_clean":"2023-10-06T04:31:35.063678+0000","last_became_active":"2023-10-06T04:31:35.060663+0000","last_became_peered":"2023-10-06T04:31:35.060663+0000","last_unstale":"2023-10-06T04:31:35.063678+0000","last_undegraded":"2023-10-06T04:31:35.063678+0000","last_fullsized":"2023-10-06T04:31:35.063678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065527+0000","last_change":"2023-10-06T04:31:35.062498+0000","last_active":"2023-10-06T04:31:35.065527+0000","last_peered":"2023-10-06T04:31:35.065527+0000","last_clean":"2023-10-06T04:31:35.065527+0000","last_became_active":"2023-10-06T04:31:35.062236+0000","last_became_peered":"2023-10-06T04:31:35.062236+0000","last_unstale":"2023-10-06T04:31:35.065527+0000","last_undegraded":"2023-10-06T04:31:35.065527+0000","last_fullsized":"2023-10-06T04:31:35.065527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059772+0000","last_change":"2023-10-06T04:31:35.057945+0000","last_active":"2023-10-06T04:31:35.059772+0000","last_peered":"2023-10-06T04:31:35.059772+0000","last_clean":"2023-10-06T04:31:35.059772+0000","last_became_active":"2023-10-06T04:31:35.057568+0000","last_became_peered":"2023-10-06T04:31:35.057568+0000","last_unstale":"2023-10-06T04:31:35.059772+0000","last_undegraded":"2023-10-06T04:31:35.059772+0000","last_fullsized":"2023-10-06T04:31:35.059772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:55.588795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057478+0000","last_change":"2023-10-06T04:31:35.051600+0000","last_active":"2023-10-06T04:31:35.057478+0000","last_peered":"2023-10-06T04:31:35.057478+0000","last_clean":"2023-10-06T04:31:35.057478+0000","last_became_active":"2023-10-06T04:31:35.051344+0000","last_became_peered":"2023-10-06T04:31:35.051344+0000","last_unstale":"2023-10-06T04:31:35.057478+0000","last_undegraded":"2023-10-06T04:31:35.057478+0000","last_fullsized":"2023-10-06T04:31:35.057478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064313+0000","last_change":"2023-10-06T04:31:35.055073+0000","last_active":"2023-10-06T04:31:35.064313+0000","last_peered":"2023-10-06T04:31:35.064313+0000","last_clean":"2023-10-06T04:31:35.064313+0000","last_became_active":"2023-10-06T04:31:35.054723+0000","last_became_peered":"2023-10-06T04:31:35.054723+0000","last_unstale":"2023-10-06T04:31:35.064313+0000","last_undegraded":"2023-10-06T04:31:35.064313+0000","last_fullsized":"2023-10-06T04:31:35.064313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:53.494026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065642+0000","last_change":"2023-10-06T04:31:35.065191+0000","last_active":"2023-10-06T04:31:35.065642+0000","last_peered":"2023-10-06T04:31:35.065642+0000","last_clean":"2023-10-06T04:31:35.065642+0000","last_became_active":"2023-10-06T04:31:35.064865+0000","last_became_peered":"2023-10-06T04:31:35.064865+0000","last_unstale":"2023-10-06T04:31:35.065642+0000","last_undegraded":"2023-10-06T04:31:35.065642+0000","last_fullsized":"2023-10-06T04:31:35.065642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.067265+0000","last_change":"2023-10-06T04:31:35.065835+0000","last_active":"2023-10-06T04:31:35.067265+0000","last_peered":"2023-10-06T04:31:35.067265+0000","last_clean":"2023-10-06T04:31:35.067265+0000","last_became_active":"2023-10-06T04:31:35.065225+0000","last_became_peered":"2023-10-06T04:31:35.065225+0000","last_unstale":"2023-10-06T04:31:35.067265+0000","last_undegraded":"2023-10-06T04:31:35.067265+0000","last_fullsized":"2023-10-06T04:31:35.067265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051966+0000","last_change":"2023-10-06T04:31:35.050260+0000","last_active":"2023-10-06T04:31:35.051966+0000","last_peered":"2023-10-06T04:31:35.051966+0000","last_clean":"2023-10-06T04:31:35.051966+0000","last_became_active":"2023-10-06T04:31:35.049943+0000","last_became_peered":"2023-10-06T04:31:35.049943+0000","last_unstale":"2023-10-06T04:31:35.051966+0000","last_undegraded":"2023-10-06T04:31:35.051966+0000","last_fullsized":"2023-10-06T04:31:35.051966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063594+0000","last_change":"2023-10-06T04:31:35.060912+0000","last_active":"2023-10-06T04:31:35.063594+0000","last_peered":"2023-10-06T04:31:35.063594+0000","last_clean":"2023-10-06T04:31:35.063594+0000","last_became_active":"2023-10-06T04:31:35.060330+0000","last_became_peered":"2023-10-06T04:31:35.060330+0000","last_unstale":"2023-10-06T04:31:35.063594+0000","last_undegraded":"2023-10-06T04:31:35.063594+0000","last_fullsized":"2023-10-06T04:31:35.063594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059719+0000","last_change":"2023-10-06T04:31:35.057646+0000","last_active":"2023-10-06T04:31:35.059719+0000","last_peered":"2023-10-06T04:31:35.059719+0000","last_clean":"2023-10-06T04:31:35.059719+0000","last_became_active":"2023-10-06T04:31:35.057375+0000","last_became_peered":"2023-10-06T04:31:35.057375+0000","last_unstale":"2023-10-06T04:31:35.059719+0000","last_undegraded":"2023-10-06T04:31:35.059719+0000","last_fullsized":"2023-10-06T04:31:35.059719+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:44.450170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.053893+0000","last_change":"2023-10-06T04:31:35.046638+0000","last_active":"2023-10-06T04:31:35.053893+0000","last_peered":"2023-10-06T04:31:35.053893+0000","last_clean":"2023-10-06T04:31:35.053893+0000","last_became_active":"2023-10-06T04:31:35.046376+0000","last_became_peered":"2023-10-06T04:31:35.046376+0000","last_unstale":"2023-10-06T04:31:35.053893+0000","last_undegraded":"2023-10-06T04:31:35.053893+0000","last_fullsized":"2023-10-06T04:31:35.053893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:33.326007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054122+0000","last_change":"2023-10-06T04:31:35.052597+0000","last_active":"2023-10-06T04:31:35.054122+0000","last_peered":"2023-10-06T04:31:35.054122+0000","last_clean":"2023-10-06T04:31:35.054122+0000","last_became_active":"2023-10-06T04:31:35.052318+0000","last_became_peered":"2023-10-06T04:31:35.052318+0000","last_unstale":"2023-10-06T04:31:35.054122+0000","last_undegraded":"2023-10-06T04:31:35.054122+0000","last_fullsized":"2023-10-06T04:31:35.054122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.068060+0000","last_change":"2023-10-06T04:31:35.067001+0000","last_active":"2023-10-06T04:31:35.068060+0000","last_peered":"2023-10-06T04:31:35.068060+0000","last_clean":"2023-10-06T04:31:35.068060+0000","last_became_active":"2023-10-06T04:31:35.066763+0000","last_became_peered":"2023-10-06T04:31:35.066763+0000","last_unstale":"2023-10-06T04:31:35.068060+0000","last_undegraded":"2023-10-06T04:31:35.068060+0000","last_fullsized":"2023-10-06T04:31:35.068060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063467+0000","last_change":"2023-10-06T04:31:35.059383+0000","last_active":"2023-10-06T04:31:35.063467+0000","last_peered":"2023-10-06T04:31:35.063467+0000","last_clean":"2023-10-06T04:31:35.063467+0000","last_became_active":"2023-10-06T04:31:35.059106+0000","last_became_peered":"2023-10-06T04:31:35.059106+0000","last_unstale":"2023-10-06T04:31:35.063467+0000","last_undegraded":"2023-10-06T04:31:35.063467+0000","last_fullsized":"2023-10-06T04:31:35.063467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063842+0000","last_change":"2023-10-06T04:31:35.054733+0000","last_active":"2023-10-06T04:31:35.063842+0000","last_peered":"2023-10-06T04:31:35.063842+0000","last_clean":"2023-10-06T04:31:35.063842+0000","last_became_active":"2023-10-06T04:31:35.054454+0000","last_became_peered":"2023-10-06T04:31:35.054454+0000","last_unstale":"2023-10-06T04:31:35.063842+0000","last_undegraded":"2023-10-06T04:31:35.063842+0000","last_fullsized":"2023-10-06T04:31:35.063842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061339+0000","last_change":"2023-10-06T04:31:35.059648+0000","last_active":"2023-10-06T04:31:35.061339+0000","last_peered":"2023-10-06T04:31:35.061339+0000","last_clean":"2023-10-06T04:31:35.061339+0000","last_became_active":"2023-10-06T04:31:35.059380+0000","last_became_peered":"2023-10-06T04:31:35.059380+0000","last_unstale":"2023-10-06T04:31:35.061339+0000","last_undegraded":"2023-10-06T04:31:35.061339+0000","last_fullsized":"2023-10-06T04:31:35.061339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066488+0000","last_change":"2023-10-06T04:31:35.063285+0000","last_active":"2023-10-06T04:31:35.066488+0000","last_peered":"2023-10-06T04:31:35.066488+0000","last_clean":"2023-10-06T04:31:35.066488+0000","last_became_active":"2023-10-06T04:31:35.062975+0000","last_became_peered":"2023-10-06T04:31:35.062975+0000","last_unstale":"2023-10-06T04:31:35.066488+0000","last_undegraded":"2023-10-06T04:31:35.066488+0000","last_fullsized":"2023-10-06T04:31:35.066488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063967+0000","last_change":"2023-10-06T04:31:35.058231+0000","last_active":"2023-10-06T04:31:35.063967+0000","last_peered":"2023-10-06T04:31:35.063967+0000","last_clean":"2023-10-06T04:31:35.063967+0000","last_became_active":"2023-10-06T04:31:35.057944+0000","last_became_peered":"2023-10-06T04:31:35.057944+0000","last_unstale":"2023-10-06T04:31:35.063967+0000","last_undegraded":"2023-10-06T04:31:35.063967+0000","last_fullsized":"2023-10-06T04:31:35.063967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065515+0000","last_change":"2023-10-06T04:31:35.063120+0000","last_active":"2023-10-06T04:31:35.065515+0000","last_peered":"2023-10-06T04:31:35.065515+0000","last_clean":"2023-10-06T04:31:35.065515+0000","last_became_active":"2023-10-06T04:31:35.062868+0000","last_became_peered":"2023-10-06T04:31:35.062868+0000","last_unstale":"2023-10-06T04:31:35.065515+0000","last_undegraded":"2023-10-06T04:31:35.065515+0000","last_fullsized":"2023-10-06T04:31:35.065515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.251995+0000","last_change":"2023-10-06T04:31:34.042620+0000","last_active":"2023-10-06T04:31:35.251995+0000","last_peered":"2023-10-06T04:31:35.251995+0000","last_clean":"2023-10-06T04:31:35.251995+0000","last_became_active":"2023-10-06T04:31:34.042023+0000","last_became_peered":"2023-10-06T04:31:34.042023+0000","last_unstale":"2023-10-06T04:31:35.251995+0000","last_undegraded":"2023-10-06T04:31:35.251995+0000","last_fullsized":"2023-10-06T04:31:35.251995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066558+0000","last_change":"2023-10-06T04:31:35.060955+0000","last_active":"2023-10-06T04:31:35.066558+0000","last_peered":"2023-10-06T04:31:35.066558+0000","last_clean":"2023-10-06T04:31:35.066558+0000","last_became_active":"2023-10-06T04:31:35.060124+0000","last_became_peered":"2023-10-06T04:31:35.060124+0000","last_unstale":"2023-10-06T04:31:35.066558+0000","last_undegraded":"2023-10-06T04:31:35.066558+0000","last_fullsized":"2023-10-06T04:31:35.066558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052353+0000","last_change":"2023-10-06T04:31:34.036811+0000","last_active":"2023-10-06T04:31:35.052353+0000","last_peered":"2023-10-06T04:31:35.052353+0000","last_clean":"2023-10-06T04:31:35.052353+0000","last_became_active":"2023-10-06T04:31:34.036513+0000","last_became_peered":"2023-10-06T04:31:34.036513+0000","last_unstale":"2023-10-06T04:31:35.052353+0000","last_undegraded":"2023-10-06T04:31:35.052353+0000","last_fullsized":"2023-10-06T04:31:35.052353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062783+0000","last_change":"2023-10-06T04:31:35.055379+0000","last_active":"2023-10-06T04:31:35.062783+0000","last_peered":"2023-10-06T04:31:35.062783+0000","last_clean":"2023-10-06T04:31:35.062783+0000","last_became_active":"2023-10-06T04:31:35.055088+0000","last_became_peered":"2023-10-06T04:31:35.055088+0000","last_unstale":"2023-10-06T04:31:35.062783+0000","last_undegraded":"2023-10-06T04:31:35.062783+0000","last_fullsized":"2023-10-06T04:31:35.062783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.048203+0000","last_change":"2023-10-06T04:31:34.054496+0000","last_active":"2023-10-06T04:31:35.048203+0000","last_peered":"2023-10-06T04:31:35.048203+0000","last_clean":"2023-10-06T04:31:35.048203+0000","last_became_active":"2023-10-06T04:31:34.054225+0000","last_became_peered":"2023-10-06T04:31:34.054225+0000","last_unstale":"2023-10-06T04:31:35.048203+0000","last_undegraded":"2023-10-06T04:31:35.048203+0000","last_fullsized":"2023-10-06T04:31:35.048203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062762+0000","last_change":"2023-10-06T04:31:35.058805+0000","last_active":"2023-10-06T04:31:35.062762+0000","last_peered":"2023-10-06T04:31:35.062762+0000","last_clean":"2023-10-06T04:31:35.062762+0000","last_became_active":"2023-10-06T04:31:35.058524+0000","last_became_peered":"2023-10-06T04:31:35.058524+0000","last_unstale":"2023-10-06T04:31:35.062762+0000","last_undegraded":"2023-10-06T04:31:35.062762+0000","last_fullsized":"2023-10-06T04:31:35.062762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046385+0000","last_change":"2023-10-06T04:31:34.042810+0000","last_active":"2023-10-06T04:31:35.046385+0000","last_peered":"2023-10-06T04:31:35.046385+0000","last_clean":"2023-10-06T04:31:35.046385+0000","last_became_active":"2023-10-06T04:31:34.040519+0000","last_became_peered":"2023-10-06T04:31:34.040519+0000","last_unstale":"2023-10-06T04:31:35.046385+0000","last_undegraded":"2023-10-06T04:31:35.046385+0000","last_fullsized":"2023-10-06T04:31:35.046385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066465+0000","last_change":"2023-10-06T04:31:35.061039+0000","last_active":"2023-10-06T04:31:35.066465+0000","last_peered":"2023-10-06T04:31:35.066465+0000","last_clean":"2023-10-06T04:31:35.066465+0000","last_became_active":"2023-10-06T04:31:35.060281+0000","last_became_peered":"2023-10-06T04:31:35.060281+0000","last_unstale":"2023-10-06T04:31:35.066465+0000","last_undegraded":"2023-10-06T04:31:35.066465+0000","last_fullsized":"2023-10-06T04:31:35.066465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.053685+0000","last_change":"2023-10-06T04:31:34.039478+0000","last_active":"2023-10-06T04:31:35.053685+0000","last_peered":"2023-10-06T04:31:35.053685+0000","last_clean":"2023-10-06T04:31:35.053685+0000","last_became_active":"2023-10-06T04:31:34.039209+0000","last_became_peered":"2023-10-06T04:31:34.039209+0000","last_unstale":"2023-10-06T04:31:35.053685+0000","last_undegraded":"2023-10-06T04:31:35.053685+0000","last_fullsized":"2023-10-06T04:31:35.053685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066712+0000","last_change":"2023-10-06T04:31:35.063717+0000","last_active":"2023-10-06T04:31:35.066712+0000","last_peered":"2023-10-06T04:31:35.066712+0000","last_clean":"2023-10-06T04:31:35.066712+0000","last_became_active":"2023-10-06T04:31:35.060241+0000","last_became_peered":"2023-10-06T04:31:35.060241+0000","last_unstale":"2023-10-06T04:31:35.066712+0000","last_undegraded":"2023-10-06T04:31:35.066712+0000","last_fullsized":"2023-10-06T04:31:35.066712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.254772+0000","last_change":"2023-10-06T04:31:34.045777+0000","last_active":"2023-10-06T04:31:35.254772+0000","last_peered":"2023-10-06T04:31:35.254772+0000","last_clean":"2023-10-06T04:31:35.254772+0000","last_became_active":"2023-10-06T04:31:34.045498+0000","last_became_peered":"2023-10-06T04:31:34.045498+0000","last_unstale":"2023-10-06T04:31:35.254772+0000","last_undegraded":"2023-10-06T04:31:35.254772+0000","last_fullsized":"2023-10-06T04:31:35.254772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.139941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066500+0000","last_change":"2023-10-06T04:31:35.063439+0000","last_active":"2023-10-06T04:31:35.066500+0000","last_peered":"2023-10-06T04:31:35.066500+0000","last_clean":"2023-10-06T04:31:35.066500+0000","last_became_active":"2023-10-06T04:31:35.063117+0000","last_became_peered":"2023-10-06T04:31:35.063117+0000","last_unstale":"2023-10-06T04:31:35.066500+0000","last_undegraded":"2023-10-06T04:31:35.066500+0000","last_fullsized":"2023-10-06T04:31:35.066500+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:16.144634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.260205+0000","last_change":"2023-10-06T04:31:34.039091+0000","last_active":"2023-10-06T04:31:35.260205+0000","last_peered":"2023-10-06T04:31:35.260205+0000","last_clean":"2023-10-06T04:31:35.260205+0000","last_became_active":"2023-10-06T04:31:34.038778+0000","last_became_peered":"2023-10-06T04:31:34.038778+0000","last_unstale":"2023-10-06T04:31:35.260205+0000","last_undegraded":"2023-10-06T04:31:35.260205+0000","last_fullsized":"2023-10-06T04:31:35.260205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058518+0000","last_change":"2023-10-06T04:31:35.052851+0000","last_active":"2023-10-06T04:31:35.058518+0000","last_peered":"2023-10-06T04:31:35.058518+0000","last_clean":"2023-10-06T04:31:35.058518+0000","last_became_active":"2023-10-06T04:31:35.052560+0000","last_became_peered":"2023-10-06T04:31:35.052560+0000","last_unstale":"2023-10-06T04:31:35.058518+0000","last_undegraded":"2023-10-06T04:31:35.058518+0000","last_fullsized":"2023-10-06T04:31:35.058518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:39.105270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255046+0000","last_change":"2023-10-06T04:31:34.046053+0000","last_active":"2023-10-06T04:31:35.255046+0000","last_peered":"2023-10-06T04:31:35.255046+0000","last_clean":"2023-10-06T04:31:35.255046+0000","last_became_active":"2023-10-06T04:31:34.045776+0000","last_became_peered":"2023-10-06T04:31:34.045776+0000","last_unstale":"2023-10-06T04:31:35.255046+0000","last_undegraded":"2023-10-06T04:31:35.255046+0000","last_fullsized":"2023-10-06T04:31:35.255046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:40.537300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066602+0000","last_change":"2023-10-06T04:31:35.062504+0000","last_active":"2023-10-06T04:31:35.066602+0000","last_peered":"2023-10-06T04:31:35.066602+0000","last_clean":"2023-10-06T04:31:35.066602+0000","last_became_active":"2023-10-06T04:31:35.060884+0000","last_became_peered":"2023-10-06T04:31:35.060884+0000","last_unstale":"2023-10-06T04:31:35.066602+0000","last_undegraded":"2023-10-06T04:31:35.066602+0000","last_fullsized":"2023-10-06T04:31:35.066602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.045014+0000","last_change":"2023-10-06T04:31:34.042234+0000","last_active":"2023-10-06T04:31:35.045014+0000","last_peered":"2023-10-06T04:31:35.045014+0000","last_clean":"2023-10-06T04:31:35.045014+0000","last_became_active":"2023-10-06T04:31:34.041959+0000","last_became_peered":"2023-10-06T04:31:34.041959+0000","last_unstale":"2023-10-06T04:31:35.045014+0000","last_undegraded":"2023-10-06T04:31:35.045014+0000","last_fullsized":"2023-10-06T04:31:35.045014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064999+0000","last_change":"2023-10-06T04:31:35.061678+0000","last_active":"2023-10-06T04:31:35.064999+0000","last_peered":"2023-10-06T04:31:35.064999+0000","last_clean":"2023-10-06T04:31:35.064999+0000","last_became_active":"2023-10-06T04:31:35.061383+0000","last_became_peered":"2023-10-06T04:31:35.061383+0000","last_unstale":"2023-10-06T04:31:35.064999+0000","last_undegraded":"2023-10-06T04:31:35.064999+0000","last_fullsized":"2023-10-06T04:31:35.064999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051432+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:35.051432+0000","last_peered":"2023-10-06T04:31:35.051432+0000","last_clean":"2023-10-06T04:31:35.051432+0000","last_became_active":"2023-10-06T04:31:34.046107+0000","last_became_peered":"2023-10-06T04:31:34.046107+0000","last_unstale":"2023-10-06T04:31:35.051432+0000","last_undegraded":"2023-10-06T04:31:35.051432+0000","last_fullsized":"2023-10-06T04:31:35.051432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062761+0000","last_change":"2023-10-06T04:31:35.055168+0000","last_active":"2023-10-06T04:31:35.062761+0000","last_peered":"2023-10-06T04:31:35.062761+0000","last_clean":"2023-10-06T04:31:35.062761+0000","last_became_active":"2023-10-06T04:31:35.054903+0000","last_became_peered":"2023-10-06T04:31:35.054903+0000","last_unstale":"2023-10-06T04:31:35.062761+0000","last_undegraded":"2023-10-06T04:31:35.062761+0000","last_fullsized":"2023-10-06T04:31:35.062761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257527+0000","last_change":"2023-10-06T04:31:34.039233+0000","last_active":"2023-10-06T04:31:35.257527+0000","last_peered":"2023-10-06T04:31:35.257527+0000","last_clean":"2023-10-06T04:31:35.257527+0000","last_became_active":"2023-10-06T04:31:34.038916+0000","last_became_peered":"2023-10-06T04:31:34.038916+0000","last_unstale":"2023-10-06T04:31:35.257527+0000","last_undegraded":"2023-10-06T04:31:35.257527+0000","last_fullsized":"2023-10-06T04:31:35.257527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:31.215430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066585+0000","last_change":"2023-10-06T04:31:35.062145+0000","last_active":"2023-10-06T04:31:35.066585+0000","last_peered":"2023-10-06T04:31:35.066585+0000","last_clean":"2023-10-06T04:31:35.066585+0000","last_became_active":"2023-10-06T04:31:35.061180+0000","last_became_peered":"2023-10-06T04:31:35.061180+0000","last_unstale":"2023-10-06T04:31:35.066585+0000","last_undegraded":"2023-10-06T04:31:35.066585+0000","last_fullsized":"2023-10-06T04:31:35.066585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255042+0000","last_change":"2023-10-06T04:31:34.043099+0000","last_active":"2023-10-06T04:31:35.255042+0000","last_peered":"2023-10-06T04:31:35.255042+0000","last_clean":"2023-10-06T04:31:35.255042+0000","last_became_active":"2023-10-06T04:31:34.042869+0000","last_became_peered":"2023-10-06T04:31:34.042869+0000","last_unstale":"2023-10-06T04:31:35.255042+0000","last_undegraded":"2023-10-06T04:31:35.255042+0000","last_fullsized":"2023-10-06T04:31:35.255042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065006+0000","last_change":"2023-10-06T04:31:35.061606+0000","last_active":"2023-10-06T04:31:35.065006+0000","last_peered":"2023-10-06T04:31:35.065006+0000","last_clean":"2023-10-06T04:31:35.065006+0000","last_became_active":"2023-10-06T04:31:35.061333+0000","last_became_peered":"2023-10-06T04:31:35.061333+0000","last_unstale":"2023-10-06T04:31:35.065006+0000","last_undegraded":"2023-10-06T04:31:35.065006+0000","last_fullsized":"2023-10-06T04:31:35.065006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255162+0000","last_change":"2023-10-06T04:31:34.045981+0000","last_active":"2023-10-06T04:31:35.255162+0000","last_peered":"2023-10-06T04:31:35.255162+0000","last_clean":"2023-10-06T04:31:35.255162+0000","last_became_active":"2023-10-06T04:31:34.045683+0000","last_became_peered":"2023-10-06T04:31:34.045683+0000","last_unstale":"2023-10-06T04:31:35.255162+0000","last_undegraded":"2023-10-06T04:31:35.255162+0000","last_fullsized":"2023-10-06T04:31:35.255162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:22.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062438+0000","last_change":"2023-10-06T04:31:35.059429+0000","last_active":"2023-10-06T04:31:35.062438+0000","last_peered":"2023-10-06T04:31:35.062438+0000","last_clean":"2023-10-06T04:31:35.062438+0000","last_became_active":"2023-10-06T04:31:35.058829+0000","last_became_peered":"2023-10-06T04:31:35.058829+0000","last_unstale":"2023-10-06T04:31:35.062438+0000","last_undegraded":"2023-10-06T04:31:35.062438+0000","last_fullsized":"2023-10-06T04:31:35.062438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257596+0000","last_change":"2023-10-06T04:31:34.048038+0000","last_active":"2023-10-06T04:31:35.257596+0000","last_peered":"2023-10-06T04:31:35.257596+0000","last_clean":"2023-10-06T04:31:35.257596+0000","last_became_active":"2023-10-06T04:31:34.047780+0000","last_became_peered":"2023-10-06T04:31:34.047780+0000","last_unstale":"2023-10-06T04:31:35.257596+0000","last_undegraded":"2023-10-06T04:31:35.257596+0000","last_fullsized":"2023-10-06T04:31:35.257596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:21.519366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060134+0000","last_change":"2023-10-06T04:31:35.057281+0000","last_active":"2023-10-06T04:31:35.060134+0000","last_peered":"2023-10-06T04:31:35.060134+0000","last_clean":"2023-10-06T04:31:35.060134+0000","last_became_active":"2023-10-06T04:31:35.056855+0000","last_became_peered":"2023-10-06T04:31:35.056855+0000","last_unstale":"2023-10-06T04:31:35.060134+0000","last_undegraded":"2023-10-06T04:31:35.060134+0000","last_fullsized":"2023-10-06T04:31:35.060134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:23.765556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051180+0000","last_change":"2023-10-06T04:31:34.046008+0000","last_active":"2023-10-06T04:31:35.051180+0000","last_peered":"2023-10-06T04:31:35.051180+0000","last_clean":"2023-10-06T04:31:35.051180+0000","last_became_active":"2023-10-06T04:31:34.045732+0000","last_became_peered":"2023-10-06T04:31:34.045732+0000","last_unstale":"2023-10-06T04:31:35.051180+0000","last_undegraded":"2023-10-06T04:31:35.051180+0000","last_fullsized":"2023-10-06T04:31:35.051180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063405+0000","last_change":"2023-10-06T04:31:35.059341+0000","last_active":"2023-10-06T04:31:35.063405+0000","last_peered":"2023-10-06T04:31:35.063405+0000","last_clean":"2023-10-06T04:31:35.063405+0000","last_became_active":"2023-10-06T04:31:35.059087+0000","last_became_peered":"2023-10-06T04:31:35.059087+0000","last_unstale":"2023-10-06T04:31:35.063405+0000","last_undegraded":"2023-10-06T04:31:35.063405+0000","last_fullsized":"2023-10-06T04:31:35.063405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.249472+0000","last_change":"2023-10-06T04:31:34.045304+0000","last_active":"2023-10-06T04:31:35.249472+0000","last_peered":"2023-10-06T04:31:35.249472+0000","last_clean":"2023-10-06T04:31:35.249472+0000","last_became_active":"2023-10-06T04:31:34.044891+0000","last_became_peered":"2023-10-06T04:31:34.044891+0000","last_unstale":"2023-10-06T04:31:35.249472+0000","last_undegraded":"2023-10-06T04:31:35.249472+0000","last_fullsized":"2023-10-06T04:31:35.249472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059237+0000","last_change":"2023-10-06T04:31:35.055035+0000","last_active":"2023-10-06T04:31:35.059237+0000","last_peered":"2023-10-06T04:31:35.059237+0000","last_clean":"2023-10-06T04:31:35.059237+0000","last_became_active":"2023-10-06T04:31:35.054727+0000","last_became_peered":"2023-10-06T04:31:35.054727+0000","last_unstale":"2023-10-06T04:31:35.059237+0000","last_undegraded":"2023-10-06T04:31:35.059237+0000","last_fullsized":"2023-10-06T04:31:35.059237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:31.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.043967+0000","last_change":"2023-10-06T04:31:34.041723+0000","last_active":"2023-10-06T04:31:35.043967+0000","last_peered":"2023-10-06T04:31:35.043967+0000","last_clean":"2023-10-06T04:31:35.043967+0000","last_became_active":"2023-10-06T04:31:34.041386+0000","last_became_peered":"2023-10-06T04:31:34.041386+0000","last_unstale":"2023-10-06T04:31:35.043967+0000","last_undegraded":"2023-10-06T04:31:35.043967+0000","last_fullsized":"2023-10-06T04:31:35.043967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:58.993061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059756+0000","last_change":"2023-10-06T04:31:35.057831+0000","last_active":"2023-10-06T04:31:35.059756+0000","last_peered":"2023-10-06T04:31:35.059756+0000","last_clean":"2023-10-06T04:31:35.059756+0000","last_became_active":"2023-10-06T04:31:35.057257+0000","last_became_peered":"2023-10-06T04:31:35.057257+0000","last_unstale":"2023-10-06T04:31:35.059756+0000","last_undegraded":"2023-10-06T04:31:35.059756+0000","last_fullsized":"2023-10-06T04:31:35.059756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046478+0000","last_change":"2023-10-06T04:31:34.035016+0000","last_active":"2023-10-06T04:31:35.046478+0000","last_peered":"2023-10-06T04:31:35.046478+0000","last_clean":"2023-10-06T04:31:35.046478+0000","last_became_active":"2023-10-06T04:31:34.034721+0000","last_became_peered":"2023-10-06T04:31:34.034721+0000","last_unstale":"2023-10-06T04:31:35.046478+0000","last_undegraded":"2023-10-06T04:31:35.046478+0000","last_fullsized":"2023-10-06T04:31:35.046478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051940+0000","last_change":"2023-10-06T04:31:35.049984+0000","last_active":"2023-10-06T04:31:35.051940+0000","last_peered":"2023-10-06T04:31:35.051940+0000","last_clean":"2023-10-06T04:31:35.051940+0000","last_became_active":"2023-10-06T04:31:35.049687+0000","last_became_peered":"2023-10-06T04:31:35.049687+0000","last_unstale":"2023-10-06T04:31:35.051940+0000","last_undegraded":"2023-10-06T04:31:35.051940+0000","last_fullsized":"2023-10-06T04:31:35.051940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.048258+0000","last_change":"2023-10-06T04:31:34.046626+0000","last_active":"2023-10-06T04:31:35.048258+0000","last_peered":"2023-10-06T04:31:35.048258+0000","last_clean":"2023-10-06T04:31:35.048258+0000","last_became_active":"2023-10-06T04:31:34.044656+0000","last_became_peered":"2023-10-06T04:31:34.044656+0000","last_unstale":"2023-10-06T04:31:35.048258+0000","last_undegraded":"2023-10-06T04:31:35.048258+0000","last_fullsized":"2023-10-06T04:31:35.048258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064326+0000","last_change":"2023-10-06T04:31:35.059496+0000","last_active":"2023-10-06T04:31:35.064326+0000","last_peered":"2023-10-06T04:31:35.064326+0000","last_clean":"2023-10-06T04:31:35.064326+0000","last_became_active":"2023-10-06T04:31:35.059187+0000","last_became_peered":"2023-10-06T04:31:35.059187+0000","last_unstale":"2023-10-06T04:31:35.064326+0000","last_undegraded":"2023-10-06T04:31:35.064326+0000","last_fullsized":"2023-10-06T04:31:35.064326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052813+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:31:35.052813+0000","last_peered":"2023-10-06T04:31:35.052813+0000","last_clean":"2023-10-06T04:31:35.052813+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:31:35.052813+0000","last_undegraded":"2023-10-06T04:31:35.052813+0000","last_fullsized":"2023-10-06T04:31:35.052813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057242+0000","last_change":"2023-10-06T04:31:34.036361+0000","last_active":"2023-10-06T04:31:35.057242+0000","last_peered":"2023-10-06T04:31:35.057242+0000","last_clean":"2023-10-06T04:31:35.057242+0000","last_became_active":"2023-10-06T04:31:34.036048+0000","last_became_peered":"2023-10-06T04:31:34.036048+0000","last_unstale":"2023-10-06T04:31:35.057242+0000","last_undegraded":"2023-10-06T04:31:35.057242+0000","last_fullsized":"2023-10-06T04:31:35.057242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065390+0000","last_change":"2023-10-06T04:31:35.061294+0000","last_active":"2023-10-06T04:31:35.065390+0000","last_peered":"2023-10-06T04:31:35.065390+0000","last_clean":"2023-10-06T04:31:35.065390+0000","last_became_active":"2023-10-06T04:31:35.061034+0000","last_became_peered":"2023-10-06T04:31:35.061034+0000","last_unstale":"2023-10-06T04:31:35.065390+0000","last_undegraded":"2023-10-06T04:31:35.065390+0000","last_fullsized":"2023-10-06T04:31:35.065390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:59.355740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055363+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:31:35.055363+0000","last_peered":"2023-10-06T04:31:35.055363+0000","last_clean":"2023-10-06T04:31:35.055363+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:31:35.055363+0000","last_undegraded":"2023-10-06T04:31:35.055363+0000","last_fullsized":"2023-10-06T04:31:35.055363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046081+0000","last_change":"2023-10-06T04:31:34.043758+0000","last_active":"2023-10-06T04:31:35.046081+0000","last_peered":"2023-10-06T04:31:35.046081+0000","last_clean":"2023-10-06T04:31:35.046081+0000","last_became_active":"2023-10-06T04:31:34.043386+0000","last_became_peered":"2023-10-06T04:31:34.043386+0000","last_unstale":"2023-10-06T04:31:35.046081+0000","last_undegraded":"2023-10-06T04:31:35.046081+0000","last_fullsized":"2023-10-06T04:31:35.046081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064019+0000","last_change":"2023-10-06T04:31:35.058248+0000","last_active":"2023-10-06T04:31:35.064019+0000","last_peered":"2023-10-06T04:31:35.064019+0000","last_clean":"2023-10-06T04:31:35.064019+0000","last_became_active":"2023-10-06T04:31:35.057955+0000","last_became_peered":"2023-10-06T04:31:35.057955+0000","last_unstale":"2023-10-06T04:31:35.064019+0000","last_undegraded":"2023-10-06T04:31:35.064019+0000","last_fullsized":"2023-10-06T04:31:35.064019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.040377+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:31:35.040377+0000","last_peered":"2023-10-06T04:31:35.040377+0000","last_clean":"2023-10-06T04:31:35.040377+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:31:35.040377+0000","last_undegraded":"2023-10-06T04:31:35.040377+0000","last_fullsized":"2023-10-06T04:31:35.040377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052376+0000","last_change":"2023-10-06T04:31:34.041900+0000","last_active":"2023-10-06T04:31:35.052376+0000","last_peered":"2023-10-06T04:31:35.052376+0000","last_clean":"2023-10-06T04:31:35.052376+0000","last_became_active":"2023-10-06T04:31:34.041621+0000","last_became_peered":"2023-10-06T04:31:34.041621+0000","last_unstale":"2023-10-06T04:31:35.052376+0000","last_undegraded":"2023-10-06T04:31:35.052376+0000","last_fullsized":"2023-10-06T04:31:35.052376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064339+0000","last_change":"2023-10-06T04:31:35.060634+0000","last_active":"2023-10-06T04:31:35.064339+0000","last_peered":"2023-10-06T04:31:35.064339+0000","last_clean":"2023-10-06T04:31:35.064339+0000","last_became_active":"2023-10-06T04:31:35.060183+0000","last_became_peered":"2023-10-06T04:31:35.060183+0000","last_unstale":"2023-10-06T04:31:35.064339+0000","last_undegraded":"2023-10-06T04:31:35.064339+0000","last_fullsized":"2023-10-06T04:31:35.064339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055245+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:31:35.055245+0000","last_peered":"2023-10-06T04:31:35.055245+0000","last_clean":"2023-10-06T04:31:35.055245+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:31:35.055245+0000","last_undegraded":"2023-10-06T04:31:35.055245+0000","last_fullsized":"2023-10-06T04:31:35.055245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049727+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:31:35.049727+0000","last_peered":"2023-10-06T04:31:35.049727+0000","last_clean":"2023-10-06T04:31:35.049727+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:31:35.049727+0000","last_undegraded":"2023-10-06T04:31:35.049727+0000","last_fullsized":"2023-10-06T04:31:35.049727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.048092+0000","last_change":"2023-10-06T04:31:34.039806+0000","last_active":"2023-10-06T04:31:35.048092+0000","last_peered":"2023-10-06T04:31:35.048092+0000","last_clean":"2023-10-06T04:31:35.048092+0000","last_became_active":"2023-10-06T04:31:34.039496+0000","last_became_peered":"2023-10-06T04:31:34.039496+0000","last_unstale":"2023-10-06T04:31:35.048092+0000","last_undegraded":"2023-10-06T04:31:35.048092+0000","last_fullsized":"2023-10-06T04:31:35.048092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052446+0000","last_change":"2023-10-06T04:31:35.050727+0000","last_active":"2023-10-06T04:31:35.052446+0000","last_peered":"2023-10-06T04:31:35.052446+0000","last_clean":"2023-10-06T04:31:35.052446+0000","last_became_active":"2023-10-06T04:31:35.050446+0000","last_became_peered":"2023-10-06T04:31:35.050446+0000","last_unstale":"2023-10-06T04:31:35.052446+0000","last_undegraded":"2023-10-06T04:31:35.052446+0000","last_fullsized":"2023-10-06T04:31:35.052446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:08.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050191+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:31:35.050191+0000","last_peered":"2023-10-06T04:31:35.050191+0000","last_clean":"2023-10-06T04:31:35.050191+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:31:35.050191+0000","last_undegraded":"2023-10-06T04:31:35.050191+0000","last_fullsized":"2023-10-06T04:31:35.050191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.044071+0000","last_change":"2023-10-06T04:31:34.039113+0000","last_active":"2023-10-06T04:31:35.044071+0000","last_peered":"2023-10-06T04:31:35.044071+0000","last_clean":"2023-10-06T04:31:35.044071+0000","last_became_active":"2023-10-06T04:31:34.038410+0000","last_became_peered":"2023-10-06T04:31:34.038410+0000","last_unstale":"2023-10-06T04:31:35.044071+0000","last_undegraded":"2023-10-06T04:31:35.044071+0000","last_fullsized":"2023-10-06T04:31:35.044071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063675+0000","last_change":"2023-10-06T04:31:35.059901+0000","last_active":"2023-10-06T04:31:35.063675+0000","last_peered":"2023-10-06T04:31:35.063675+0000","last_clean":"2023-10-06T04:31:35.063675+0000","last_became_active":"2023-10-06T04:31:35.059461+0000","last_became_peered":"2023-10-06T04:31:35.059461+0000","last_unstale":"2023-10-06T04:31:35.063675+0000","last_undegraded":"2023-10-06T04:31:35.063675+0000","last_fullsized":"2023-10-06T04:31:35.063675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049407+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:31:35.049407+0000","last_peered":"2023-10-06T04:31:35.049407+0000","last_clean":"2023-10-06T04:31:35.049407+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:31:35.049407+0000","last_undegraded":"2023-10-06T04:31:35.049407+0000","last_fullsized":"2023-10-06T04:31:35.049407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052921+0000","last_change":"2023-10-06T04:31:34.042477+0000","last_active":"2023-10-06T04:31:35.052921+0000","last_peered":"2023-10-06T04:31:35.052921+0000","last_clean":"2023-10-06T04:31:35.052921+0000","last_became_active":"2023-10-06T04:31:34.041582+0000","last_became_peered":"2023-10-06T04:31:34.041582+0000","last_unstale":"2023-10-06T04:31:35.052921+0000","last_undegraded":"2023-10-06T04:31:35.052921+0000","last_fullsized":"2023-10-06T04:31:35.052921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063792+0000","last_change":"2023-10-06T04:31:35.061904+0000","last_active":"2023-10-06T04:31:35.063792+0000","last_peered":"2023-10-06T04:31:35.063792+0000","last_clean":"2023-10-06T04:31:35.063792+0000","last_became_active":"2023-10-06T04:31:35.061655+0000","last_became_peered":"2023-10-06T04:31:35.061655+0000","last_unstale":"2023-10-06T04:31:35.063792+0000","last_undegraded":"2023-10-06T04:31:35.063792+0000","last_fullsized":"2023-10-06T04:31:35.063792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255298+0000","last_change":"2023-10-06T04:31:34.042617+0000","last_active":"2023-10-06T04:31:35.255298+0000","last_peered":"2023-10-06T04:31:35.255298+0000","last_clean":"2023-10-06T04:31:35.255298+0000","last_became_active":"2023-10-06T04:31:34.042346+0000","last_became_peered":"2023-10-06T04:31:34.042346+0000","last_unstale":"2023-10-06T04:31:35.255298+0000","last_undegraded":"2023-10-06T04:31:35.255298+0000","last_fullsized":"2023-10-06T04:31:35.255298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052516+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:31:35.052516+0000","last_peered":"2023-10-06T04:31:35.052516+0000","last_clean":"2023-10-06T04:31:35.052516+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:31:35.052516+0000","last_undegraded":"2023-10-06T04:31:35.052516+0000","last_fullsized":"2023-10-06T04:31:35.052516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066441+0000","last_change":"2023-10-06T04:31:35.060668+0000","last_active":"2023-10-06T04:31:35.066441+0000","last_peered":"2023-10-06T04:31:35.066441+0000","last_clean":"2023-10-06T04:31:35.066441+0000","last_became_active":"2023-10-06T04:31:35.059927+0000","last_became_peered":"2023-10-06T04:31:35.059927+0000","last_unstale":"2023-10-06T04:31:35.066441+0000","last_undegraded":"2023-10-06T04:31:35.066441+0000","last_fullsized":"2023-10-06T04:31:35.066441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.043872+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:31:35.043872+0000","last_peered":"2023-10-06T04:31:35.043872+0000","last_clean":"2023-10-06T04:31:35.043872+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:31:35.043872+0000","last_undegraded":"2023-10-06T04:31:35.043872+0000","last_fullsized":"2023-10-06T04:31:35.043872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T04:31:37.159225+0000","last_change":"2023-10-06T04:31:34.044177+0000","last_active":"2023-10-06T04:31:37.159225+0000","last_peered":"2023-10-06T04:31:37.159225+0000","last_clean":"2023-10-06T04:31:37.159225+0000","last_became_active":"2023-10-06T04:31:34.043921+0000","last_became_peered":"2023-10-06T04:31:34.043921+0000","last_unstale":"2023-10-06T04:31:37.159225+0000","last_undegraded":"2023-10-06T04:31:37.159225+0000","last_fullsized":"2023-10-06T04:31:37.159225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062416+0000","last_change":"2023-10-06T04:31:35.059320+0000","last_active":"2023-10-06T04:31:35.062416+0000","last_peered":"2023-10-06T04:31:35.062416+0000","last_clean":"2023-10-06T04:31:35.062416+0000","last_became_active":"2023-10-06T04:31:35.058458+0000","last_became_peered":"2023-10-06T04:31:35.058458+0000","last_unstale":"2023-10-06T04:31:35.062416+0000","last_undegraded":"2023-10-06T04:31:35.062416+0000","last_fullsized":"2023-10-06T04:31:35.062416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:06.614824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.252891+0000","last_change":"2023-10-06T04:31:34.043203+0000","last_active":"2023-10-06T04:31:35.252891+0000","last_peered":"2023-10-06T04:31:35.252891+0000","last_clean":"2023-10-06T04:31:35.252891+0000","last_became_active":"2023-10-06T04:31:34.042886+0000","last_became_peered":"2023-10-06T04:31:34.042886+0000","last_unstale":"2023-10-06T04:31:35.252891+0000","last_undegraded":"2023-10-06T04:31:35.252891+0000","last_fullsized":"2023-10-06T04:31:35.252891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066541+0000","last_change":"2023-10-06T04:31:35.057021+0000","last_active":"2023-10-06T04:31:35.066541+0000","last_peered":"2023-10-06T04:31:35.066541+0000","last_clean":"2023-10-06T04:31:35.066541+0000","last_became_active":"2023-10-06T04:31:35.056770+0000","last_became_peered":"2023-10-06T04:31:35.056770+0000","last_unstale":"2023-10-06T04:31:35.066541+0000","last_undegraded":"2023-10-06T04:31:35.066541+0000","last_fullsized":"2023-10-06T04:31:35.066541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255172+0000","last_change":"2023-10-06T04:31:34.044080+0000","last_active":"2023-10-06T04:31:35.255172+0000","last_peered":"2023-10-06T04:31:35.255172+0000","last_clean":"2023-10-06T04:31:35.255172+0000","last_became_active":"2023-10-06T04:31:34.042225+0000","last_became_peered":"2023-10-06T04:31:34.042225+0000","last_unstale":"2023-10-06T04:31:35.255172+0000","last_undegraded":"2023-10-06T04:31:35.255172+0000","last_fullsized":"2023-10-06T04:31:35.255172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:17.059659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059100+0000","last_change":"2023-10-06T04:31:35.055021+0000","last_active":"2023-10-06T04:31:35.059100+0000","last_peered":"2023-10-06T04:31:35.059100+0000","last_clean":"2023-10-06T04:31:35.059100+0000","last_became_active":"2023-10-06T04:31:35.054737+0000","last_became_peered":"2023-10-06T04:31:35.054737+0000","last_unstale":"2023-10-06T04:31:35.059100+0000","last_undegraded":"2023-10-06T04:31:35.059100+0000","last_fullsized":"2023-10-06T04:31:35.059100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:26.087138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257301+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:35.257301+0000","last_peered":"2023-10-06T04:31:35.257301+0000","last_clean":"2023-10-06T04:31:35.257301+0000","last_became_active":"2023-10-06T04:31:34.041996+0000","last_became_peered":"2023-10-06T04:31:34.041996+0000","last_unstale":"2023-10-06T04:31:35.257301+0000","last_undegraded":"2023-10-06T04:31:35.257301+0000","last_fullsized":"2023-10-06T04:31:35.257301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062379+0000","last_change":"2023-10-06T04:31:35.059718+0000","last_active":"2023-10-06T04:31:35.062379+0000","last_peered":"2023-10-06T04:31:35.062379+0000","last_clean":"2023-10-06T04:31:35.062379+0000","last_became_active":"2023-10-06T04:31:35.059438+0000","last_became_peered":"2023-10-06T04:31:35.059438+0000","last_unstale":"2023-10-06T04:31:35.062379+0000","last_undegraded":"2023-10-06T04:31:35.062379+0000","last_fullsized":"2023-10-06T04:31:35.062379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.252417+0000","last_change":"2023-10-06T04:31:34.047689+0000","last_active":"2023-10-06T04:31:35.252417+0000","last_peered":"2023-10-06T04:31:35.252417+0000","last_clean":"2023-10-06T04:31:35.252417+0000","last_became_active":"2023-10-06T04:31:34.047311+0000","last_became_peered":"2023-10-06T04:31:34.047311+0000","last_unstale":"2023-10-06T04:31:35.252417+0000","last_undegraded":"2023-10-06T04:31:35.252417+0000","last_fullsized":"2023-10-06T04:31:35.252417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063861+0000","last_change":"2023-10-06T04:31:35.055295+0000","last_active":"2023-10-06T04:31:35.063861+0000","last_peered":"2023-10-06T04:31:35.063861+0000","last_clean":"2023-10-06T04:31:35.063861+0000","last_became_active":"2023-10-06T04:31:35.055019+0000","last_became_peered":"2023-10-06T04:31:35.055019+0000","last_unstale":"2023-10-06T04:31:35.063861+0000","last_undegraded":"2023-10-06T04:31:35.063861+0000","last_fullsized":"2023-10-06T04:31:35.063861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257663+0000","last_change":"2023-10-06T04:31:34.040784+0000","last_active":"2023-10-06T04:31:35.257663+0000","last_peered":"2023-10-06T04:31:35.257663+0000","last_clean":"2023-10-06T04:31:35.257663+0000","last_became_active":"2023-10-06T04:31:34.040127+0000","last_became_peered":"2023-10-06T04:31:34.040127+0000","last_unstale":"2023-10-06T04:31:35.257663+0000","last_undegraded":"2023-10-06T04:31:35.257663+0000","last_fullsized":"2023-10-06T04:31:35.257663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054184+0000","last_change":"2023-10-06T04:31:35.052827+0000","last_active":"2023-10-06T04:31:35.054184+0000","last_peered":"2023-10-06T04:31:35.054184+0000","last_clean":"2023-10-06T04:31:35.054184+0000","last_became_active":"2023-10-06T04:31:35.052539+0000","last_became_peered":"2023-10-06T04:31:35.052539+0000","last_unstale":"2023-10-06T04:31:35.054184+0000","last_undegraded":"2023-10-06T04:31:35.054184+0000","last_fullsized":"2023-10-06T04:31:35.054184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054464+0000","last_change":"2023-10-06T04:31:35.051004+0000","last_active":"2023-10-06T04:31:35.054464+0000","last_peered":"2023-10-06T04:31:35.054464+0000","last_clean":"2023-10-06T04:31:35.054464+0000","last_became_active":"2023-10-06T04:31:35.050694+0000","last_became_peered":"2023-10-06T04:31:35.050694+0000","last_unstale":"2023-10-06T04:31:35.054464+0000","last_undegraded":"2023-10-06T04:31:35.054464+0000","last_fullsized":"2023-10-06T04:31:35.054464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066625+0000","last_change":"2023-10-06T04:31:35.062748+0000","last_active":"2023-10-06T04:31:35.066625+0000","last_peered":"2023-10-06T04:31:35.066625+0000","last_clean":"2023-10-06T04:31:35.066625+0000","last_became_active":"2023-10-06T04:31:35.060650+0000","last_became_peered":"2023-10-06T04:31:35.060650+0000","last_unstale":"2023-10-06T04:31:35.066625+0000","last_undegraded":"2023-10-06T04:31:35.066625+0000","last_fullsized":"2023-10-06T04:31:35.066625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054198+0000","last_change":"2023-10-06T04:31:35.052675+0000","last_active":"2023-10-06T04:31:35.054198+0000","last_peered":"2023-10-06T04:31:35.054198+0000","last_clean":"2023-10-06T04:31:35.054198+0000","last_became_active":"2023-10-06T04:31:35.052379+0000","last_became_peered":"2023-10-06T04:31:35.052379+0000","last_unstale":"2023-10-06T04:31:35.054198+0000","last_undegraded":"2023-10-06T04:31:35.054198+0000","last_fullsized":"2023-10-06T04:31:35.054198+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:40.270087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066464+0000","last_change":"2023-10-06T04:31:35.061097+0000","last_active":"2023-10-06T04:31:35.066464+0000","last_peered":"2023-10-06T04:31:35.066464+0000","last_clean":"2023-10-06T04:31:35.066464+0000","last_became_active":"2023-10-06T04:31:35.060817+0000","last_became_peered":"2023-10-06T04:31:35.060817+0000","last_unstale":"2023-10-06T04:31:35.066464+0000","last_undegraded":"2023-10-06T04:31:35.066464+0000","last_fullsized":"2023-10-06T04:31:35.066464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065474+0000","last_change":"2023-10-06T04:31:35.062003+0000","last_active":"2023-10-06T04:31:35.065474+0000","last_peered":"2023-10-06T04:31:35.065474+0000","last_clean":"2023-10-06T04:31:35.065474+0000","last_became_active":"2023-10-06T04:31:35.061553+0000","last_became_peered":"2023-10-06T04:31:35.061553+0000","last_unstale":"2023-10-06T04:31:35.065474+0000","last_undegraded":"2023-10-06T04:31:35.065474+0000","last_fullsized":"2023-10-06T04:31:35.065474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:31:44.977 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:31:45.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.125+0000 7f3fff5da700 1 Processor -- start 2023-10-06T04:31:45.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3fff5da700 1 -- start start 2023-10-06T04:31:45.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3fff5da700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3ff8068c90 0x7f3ff806e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3fff5da700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ff806e970 0x7f3ff8158f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3fff5da700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3ff8069a30 con 0x7f3ff80688c0 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3fff5da700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff806a890 con 0x7f3ff806e970 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3fff5da700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3ff8069c80 con 0x7f3ff8068c90 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3ffdb77700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3ff806e970 0x7f3ff8158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3ffd376700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3ff80688c0 0x7f3ff80c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57020/0 (socket says 172.21.15.123:57020) 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.126+0000 7f3ffd376700 1 -- 172.21.15.123:0/3817419050 learned_addr learned my addr 172.21.15.123:0/3817419050 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3ffcb75700 1 --2- 172.21.15.123:0/3817419050 >> v2:172.21.15.177:3300/0 conn(0x7f3ff8068c90 0x7f3ff806e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3ffdb77700 1 -- 172.21.15.123:0/3817419050 >> v2:172.21.15.177:3300/0 conn(0x7f3ff8068c90 msgr2=0x7f3ff806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3ffdb77700 1 --2- 172.21.15.123:0/3817419050 >> v2:172.21.15.177:3300/0 conn(0x7f3ff8068c90 0x7f3ff806e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3ffdb77700 1 -- 172.21.15.123:0/3817419050 >> v1:172.21.15.123:6789/0 conn(0x7f3ff80688c0 legacy=0x7f3ff80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3ffdb77700 1 -- 172.21.15.123:0/3817419050 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff8069f60 con 0x7f3ff806e970 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3feffff700 1 -- 172.21.15.123:0/3817419050 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3636445930 0 0) 0x7f3ff8069a30 con 0x7f3ff80688c0 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3ffcb75700 1 --2- 172.21.15.123:0/3817419050 >> v2:172.21.15.177:3300/0 conn(0x7f3ff8068c90 0x7f3ff806e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3ffdb77700 1 --2- 172.21.15.123:0/3817419050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806e970 0x7f3ff8158f70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3ff806d720 tx=0x7f3ff0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71305106f16fbd5f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3feffff700 1 -- 172.21.15.123:0/3817419050 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff000e070 con 0x7f3ff806e970 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3feffff700 1 -- 172.21.15.123:0/3817419050 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ff000bc30 con 0x7f3ff806e970 2023-10-06T04:31:45.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.127+0000 7f3feffff700 1 -- 172.21.15.123:0/3817419050 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ff0012430 con 0x7f3ff806e970 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 -- 172.21.15.123:0/3817419050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806e970 msgr2=0x7f3ff8158f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3817419050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806e970 0x7f3ff8158f70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f3ff806d720 tx=0x7f3ff0009780 comp rx=0 tx=0).stop 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 -- 172.21.15.123:0/3817419050 shutdown_connections 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3817419050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff806e970 0x7f3ff8158f70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3817419050 >> v2:172.21.15.177:3300/0 conn(0x7f3ff8068c90 0x7f3ff806e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 -- 172.21.15.123:0/3817419050 >> 172.21.15.123:0/3817419050 conn(0x7f3ff80c9c90 msgr2=0x7f3ff806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 -- 172.21.15.123:0/3817419050 shutdown_connections 2023-10-06T04:31:45.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.128+0000 7f3fff5da700 1 -- 172.21.15.123:0/3817419050 wait complete. 2023-10-06T04:31:45.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.129+0000 7f3fff5da700 1 Processor -- start 2023-10-06T04:31:45.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.129+0000 7f3fff5da700 1 -- start start 2023-10-06T04:31:45.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff80688c0 0x7f3ff8157230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ff8068c90 0x7f3ff80ae3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ff806e970 0x7f3ff80b1160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80ac830 con 0x7f3ff80688c0 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff8157770 con 0x7f3ff806e970 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3ff80c3cd0 con 0x7f3ff8068c90 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff80688c0 0x7f3ff8157230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff80688c0 0x7f3ff8157230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43492/0 (socket says 172.21.15.123:43492) 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 -- 172.21.15.123:0/3640147403 learned_addr learned my addr 172.21.15.123:0/3640147403 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffdb77700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ff806e970 0x7f3ff80b1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffcb75700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ff8068c90 0x7f3ff80ae3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 -- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ff8068c90 msgr2=0x7f3ff80ae3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ff8068c90 0x7f3ff80ae3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 -- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ff806e970 msgr2=0x7f3ff80b1160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ff806e970 0x7f3ff80b1160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 -- 172.21.15.123:0/3640147403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ff0009a20 con 0x7f3ff80688c0 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffcb75700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ff8068c90 0x7f3ff80ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3ffd376700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff80688c0 0x7f3ff8157230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c7e0 tx=0x7f3fe8007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fed7fa700 1 -- 172.21.15.123:0/3640147403 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8005b30 con 0x7f3ff80688c0 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fed7fa700 1 -- 172.21.15.123:0/3640147403 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fe8005c90 con 0x7f3ff80688c0 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3ff80c3fb0 con 0x7f3ff80688c0 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fed7fa700 1 -- 172.21.15.123:0/3640147403 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fe8007300 con 0x7f3ff80688c0 2023-10-06T04:31:45.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.130+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3ff80c4570 con 0x7f3ff80688c0 2023-10-06T04:31:45.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.131+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3ff806cc70 con 0x7f3ff80688c0 2023-10-06T04:31:45.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.132+0000 7f3fed7fa700 1 -- 172.21.15.123:0/3640147403 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3fe801b400 con 0x7f3ff80688c0 2023-10-06T04:31:45.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.132+0000 7f3fed7fa700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3fdc066f30 0x7f3fdc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.132+0000 7f3ffcb75700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3fdc066f30 0x7f3fdc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.132+0000 7f3fed7fa700 1 -- 172.21.15.123:0/3640147403 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3fe8084190 con 0x7f3ff80688c0 2023-10-06T04:31:45.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.132+0000 7f3ffcb75700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3fdc066f30 0x7f3fdc0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4006f00 tx=0x7f3ff4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.136+0000 7f3fed7fa700 1 -- 172.21.15.123:0/3640147403 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3fe8051320 con 0x7f3ff80688c0 2023-10-06T04:31:45.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.315+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3ff806a240 con 0x7f3fdc066f30 2023-10-06T04:31:45.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.323+0000 7f3fed7fa700 1 -- 172.21.15.123:0/3640147403 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+286527 (secure 0 0 0) 0x7f3ff806a240 con 0x7f3fdc066f30 2023-10-06T04:31:45.328 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:45.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3fdc066f30 msgr2=0x7f3fdc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3fdc066f30 0x7f3fdc0693e0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f3ff4006f00 tx=0x7f3ff4008040 comp rx=0 tx=0).stop 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff80688c0 msgr2=0x7f3ff8157230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff80688c0 0x7f3ff8157230 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f3fe800c7e0 tx=0x7f3fe8007df0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 shutdown_connections 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3fdc066f30 0x7f3fdc0693e0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3ff80688c0 0x7f3ff8157230 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3ff806e970 0x7f3ff80b1160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 --2- 172.21.15.123:0/3640147403 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3ff8068c90 0x7f3ff80ae3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 >> 172.21.15.123:0/3640147403 conn(0x7f3ff80c9c90 msgr2=0x7f3ff806ad20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 shutdown_connections 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.326+0000 7f3fff5da700 1 -- 172.21.15.123:0/3640147403 wait complete. 2023-10-06T04:31:45.331 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:31:45.340 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-10-06T04:31:44.891123+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2381416,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752593304,"statfs":{"total":773094113280,"available":770655543296,"internally_reserved":0,"allocated":3186688,"data_stored":1899900,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.853489"},"pg_stats":[{"pgid":"4.3f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066588+0000","last_change":"2023-10-06T04:31:35.057310+0000","last_active":"2023-10-06T04:31:35.066588+0000","last_peered":"2023-10-06T04:31:35.066588+0000","last_clean":"2023-10-06T04:31:35.066588+0000","last_became_active":"2023-10-06T04:31:35.057042+0000","last_became_peered":"2023-10-06T04:31:35.057042+0000","last_unstale":"2023-10-06T04:31:35.066588+0000","last_undegraded":"2023-10-06T04:31:35.066588+0000","last_fullsized":"2023-10-06T04:31:35.066588+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.3e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065545+0000","last_change":"2023-10-06T04:31:35.063658+0000","last_active":"2023-10-06T04:31:35.065545+0000","last_peered":"2023-10-06T04:31:35.065545+0000","last_clean":"2023-10-06T04:31:35.065545+0000","last_became_active":"2023-10-06T04:31:35.063403+0000","last_became_peered":"2023-10-06T04:31:35.063403+0000","last_unstale":"2023-10-06T04:31:35.065545+0000","last_undegraded":"2023-10-06T04:31:35.065545+0000","last_fullsized":"2023-10-06T04:31:35.065545+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.3d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059298+0000","last_change":"2023-10-06T04:31:35.056211+0000","last_active":"2023-10-06T04:31:35.059298+0000","last_peered":"2023-10-06T04:31:35.059298+0000","last_clean":"2023-10-06T04:31:35.059298+0000","last_became_active":"2023-10-06T04:31:35.055883+0000","last_became_peered":"2023-10-06T04:31:35.055883+0000","last_unstale":"2023-10-06T04:31:35.059298+0000","last_undegraded":"2023-10-06T04:31:35.059298+0000","last_fullsized":"2023-10-06T04:31:35.059298+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:47:54.893111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.3c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060097+0000","last_change":"2023-10-06T04:31:35.055601+0000","last_active":"2023-10-06T04:31:35.060097+0000","last_peered":"2023-10-06T04:31:35.060097+0000","last_clean":"2023-10-06T04:31:35.060097+0000","last_became_active":"2023-10-06T04:31:35.055325+0000","last_became_peered":"2023-10-06T04:31:35.055325+0000","last_unstale":"2023-10-06T04:31:35.060097+0000","last_undegraded":"2023-10-06T04:31:35.060097+0000","last_fullsized":"2023-10-06T04:31:35.060097+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063587+0000","last_change":"2023-10-06T04:31:35.061888+0000","last_active":"2023-10-06T04:31:35.063587+0000","last_peered":"2023-10-06T04:31:35.063587+0000","last_clean":"2023-10-06T04:31:35.063587+0000","last_became_active":"2023-10-06T04:31:35.061635+0000","last_became_peered":"2023-10-06T04:31:35.061635+0000","last_unstale":"2023-10-06T04:31:35.063587+0000","last_undegraded":"2023-10-06T04:31:35.063587+0000","last_fullsized":"2023-10-06T04:31:35.063587+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.3a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059324+0000","last_change":"2023-10-06T04:31:35.055786+0000","last_active":"2023-10-06T04:31:35.059324+0000","last_peered":"2023-10-06T04:31:35.059324+0000","last_clean":"2023-10-06T04:31:35.059324+0000","last_became_active":"2023-10-06T04:31:35.055501+0000","last_became_peered":"2023-10-06T04:31:35.055501+0000","last_unstale":"2023-10-06T04:31:35.059324+0000","last_undegraded":"2023-10-06T04:31:35.059324+0000","last_fullsized":"2023-10-06T04:31:35.059324+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:35:47.521895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.39","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066329+0000","last_change":"2023-10-06T04:31:35.065611+0000","last_active":"2023-10-06T04:31:35.066329+0000","last_peered":"2023-10-06T04:31:35.066329+0000","last_clean":"2023-10-06T04:31:35.066329+0000","last_became_active":"2023-10-06T04:31:35.064864+0000","last_became_peered":"2023-10-06T04:31:35.064864+0000","last_unstale":"2023-10-06T04:31:35.066329+0000","last_undegraded":"2023-10-06T04:31:35.066329+0000","last_fullsized":"2023-10-06T04:31:35.066329+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.38","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064924+0000","last_change":"2023-10-06T04:31:35.060662+0000","last_active":"2023-10-06T04:31:35.064924+0000","last_peered":"2023-10-06T04:31:35.064924+0000","last_clean":"2023-10-06T04:31:35.064924+0000","last_became_active":"2023-10-06T04:31:35.060371+0000","last_became_peered":"2023-10-06T04:31:35.060371+0000","last_unstale":"2023-10-06T04:31:35.064924+0000","last_undegraded":"2023-10-06T04:31:35.064924+0000","last_fullsized":"2023-10-06T04:31:35.064924+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.37","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063678+0000","last_change":"2023-10-06T04:31:35.061068+0000","last_active":"2023-10-06T04:31:35.063678+0000","last_peered":"2023-10-06T04:31:35.063678+0000","last_clean":"2023-10-06T04:31:35.063678+0000","last_became_active":"2023-10-06T04:31:35.060663+0000","last_became_peered":"2023-10-06T04:31:35.060663+0000","last_unstale":"2023-10-06T04:31:35.063678+0000","last_undegraded":"2023-10-06T04:31:35.063678+0000","last_fullsized":"2023-10-06T04:31:35.063678+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.36","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065527+0000","last_change":"2023-10-06T04:31:35.062498+0000","last_active":"2023-10-06T04:31:35.065527+0000","last_peered":"2023-10-06T04:31:35.065527+0000","last_clean":"2023-10-06T04:31:35.065527+0000","last_became_active":"2023-10-06T04:31:35.062236+0000","last_became_peered":"2023-10-06T04:31:35.062236+0000","last_unstale":"2023-10-06T04:31:35.065527+0000","last_undegraded":"2023-10-06T04:31:35.065527+0000","last_fullsized":"2023-10-06T04:31:35.065527+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.35","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059772+0000","last_change":"2023-10-06T04:31:35.057945+0000","last_active":"2023-10-06T04:31:35.059772+0000","last_peered":"2023-10-06T04:31:35.059772+0000","last_clean":"2023-10-06T04:31:35.059772+0000","last_became_active":"2023-10-06T04:31:35.057568+0000","last_became_peered":"2023-10-06T04:31:35.057568+0000","last_unstale":"2023-10-06T04:31:35.059772+0000","last_undegraded":"2023-10-06T04:31:35.059772+0000","last_fullsized":"2023-10-06T04:31:35.059772+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:09:55.588795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.34","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057478+0000","last_change":"2023-10-06T04:31:35.051600+0000","last_active":"2023-10-06T04:31:35.057478+0000","last_peered":"2023-10-06T04:31:35.057478+0000","last_clean":"2023-10-06T04:31:35.057478+0000","last_became_active":"2023-10-06T04:31:35.051344+0000","last_became_peered":"2023-10-06T04:31:35.051344+0000","last_unstale":"2023-10-06T04:31:35.057478+0000","last_undegraded":"2023-10-06T04:31:35.057478+0000","last_fullsized":"2023-10-06T04:31:35.057478+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:18.064352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.33","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064313+0000","last_change":"2023-10-06T04:31:35.055073+0000","last_active":"2023-10-06T04:31:35.064313+0000","last_peered":"2023-10-06T04:31:35.064313+0000","last_clean":"2023-10-06T04:31:35.064313+0000","last_became_active":"2023-10-06T04:31:35.054723+0000","last_became_peered":"2023-10-06T04:31:35.054723+0000","last_unstale":"2023-10-06T04:31:35.064313+0000","last_undegraded":"2023-10-06T04:31:35.064313+0000","last_fullsized":"2023-10-06T04:31:35.064313+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:53.494026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.32","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065642+0000","last_change":"2023-10-06T04:31:35.065191+0000","last_active":"2023-10-06T04:31:35.065642+0000","last_peered":"2023-10-06T04:31:35.065642+0000","last_clean":"2023-10-06T04:31:35.065642+0000","last_became_active":"2023-10-06T04:31:35.064865+0000","last_became_peered":"2023-10-06T04:31:35.064865+0000","last_unstale":"2023-10-06T04:31:35.065642+0000","last_undegraded":"2023-10-06T04:31:35.065642+0000","last_fullsized":"2023-10-06T04:31:35.065642+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.31","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.067265+0000","last_change":"2023-10-06T04:31:35.065835+0000","last_active":"2023-10-06T04:31:35.067265+0000","last_peered":"2023-10-06T04:31:35.067265+0000","last_clean":"2023-10-06T04:31:35.067265+0000","last_became_active":"2023-10-06T04:31:35.065225+0000","last_became_peered":"2023-10-06T04:31:35.065225+0000","last_unstale":"2023-10-06T04:31:35.067265+0000","last_undegraded":"2023-10-06T04:31:35.067265+0000","last_fullsized":"2023-10-06T04:31:35.067265+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.30","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051966+0000","last_change":"2023-10-06T04:31:35.050260+0000","last_active":"2023-10-06T04:31:35.051966+0000","last_peered":"2023-10-06T04:31:35.051966+0000","last_clean":"2023-10-06T04:31:35.051966+0000","last_became_active":"2023-10-06T04:31:35.049943+0000","last_became_peered":"2023-10-06T04:31:35.049943+0000","last_unstale":"2023-10-06T04:31:35.051966+0000","last_undegraded":"2023-10-06T04:31:35.051966+0000","last_fullsized":"2023-10-06T04:31:35.051966+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063594+0000","last_change":"2023-10-06T04:31:35.060912+0000","last_active":"2023-10-06T04:31:35.063594+0000","last_peered":"2023-10-06T04:31:35.063594+0000","last_clean":"2023-10-06T04:31:35.063594+0000","last_became_active":"2023-10-06T04:31:35.060330+0000","last_became_peered":"2023-10-06T04:31:35.060330+0000","last_unstale":"2023-10-06T04:31:35.063594+0000","last_undegraded":"2023-10-06T04:31:35.063594+0000","last_fullsized":"2023-10-06T04:31:35.063594+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059719+0000","last_change":"2023-10-06T04:31:35.057646+0000","last_active":"2023-10-06T04:31:35.059719+0000","last_peered":"2023-10-06T04:31:35.059719+0000","last_clean":"2023-10-06T04:31:35.059719+0000","last_became_active":"2023-10-06T04:31:35.057375+0000","last_became_peered":"2023-10-06T04:31:35.057375+0000","last_unstale":"2023-10-06T04:31:35.059719+0000","last_undegraded":"2023-10-06T04:31:35.059719+0000","last_fullsized":"2023-10-06T04:31:35.059719+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:04:44.450170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.053893+0000","last_change":"2023-10-06T04:31:35.046638+0000","last_active":"2023-10-06T04:31:35.053893+0000","last_peered":"2023-10-06T04:31:35.053893+0000","last_clean":"2023-10-06T04:31:35.053893+0000","last_became_active":"2023-10-06T04:31:35.046376+0000","last_became_peered":"2023-10-06T04:31:35.046376+0000","last_unstale":"2023-10-06T04:31:35.053893+0000","last_undegraded":"2023-10-06T04:31:35.053893+0000","last_fullsized":"2023-10-06T04:31:35.053893+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:33.326007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.2c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054122+0000","last_change":"2023-10-06T04:31:35.052597+0000","last_active":"2023-10-06T04:31:35.054122+0000","last_peered":"2023-10-06T04:31:35.054122+0000","last_clean":"2023-10-06T04:31:35.054122+0000","last_became_active":"2023-10-06T04:31:35.052318+0000","last_became_peered":"2023-10-06T04:31:35.052318+0000","last_unstale":"2023-10-06T04:31:35.054122+0000","last_undegraded":"2023-10-06T04:31:35.054122+0000","last_fullsized":"2023-10-06T04:31:35.054122+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.2b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.068060+0000","last_change":"2023-10-06T04:31:35.067001+0000","last_active":"2023-10-06T04:31:35.068060+0000","last_peered":"2023-10-06T04:31:35.068060+0000","last_clean":"2023-10-06T04:31:35.068060+0000","last_became_active":"2023-10-06T04:31:35.066763+0000","last_became_peered":"2023-10-06T04:31:35.066763+0000","last_unstale":"2023-10-06T04:31:35.068060+0000","last_undegraded":"2023-10-06T04:31:35.068060+0000","last_fullsized":"2023-10-06T04:31:35.068060+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.2a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063467+0000","last_change":"2023-10-06T04:31:35.059383+0000","last_active":"2023-10-06T04:31:35.063467+0000","last_peered":"2023-10-06T04:31:35.063467+0000","last_clean":"2023-10-06T04:31:35.063467+0000","last_became_active":"2023-10-06T04:31:35.059106+0000","last_became_peered":"2023-10-06T04:31:35.059106+0000","last_unstale":"2023-10-06T04:31:35.063467+0000","last_undegraded":"2023-10-06T04:31:35.063467+0000","last_fullsized":"2023-10-06T04:31:35.063467+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.29","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063842+0000","last_change":"2023-10-06T04:31:35.054733+0000","last_active":"2023-10-06T04:31:35.063842+0000","last_peered":"2023-10-06T04:31:35.063842+0000","last_clean":"2023-10-06T04:31:35.063842+0000","last_became_active":"2023-10-06T04:31:35.054454+0000","last_became_peered":"2023-10-06T04:31:35.054454+0000","last_unstale":"2023-10-06T04:31:35.063842+0000","last_undegraded":"2023-10-06T04:31:35.063842+0000","last_fullsized":"2023-10-06T04:31:35.063842+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:54:32.300959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.28","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.061339+0000","last_change":"2023-10-06T04:31:35.059648+0000","last_active":"2023-10-06T04:31:35.061339+0000","last_peered":"2023-10-06T04:31:35.061339+0000","last_clean":"2023-10-06T04:31:35.061339+0000","last_became_active":"2023-10-06T04:31:35.059380+0000","last_became_peered":"2023-10-06T04:31:35.059380+0000","last_unstale":"2023-10-06T04:31:35.061339+0000","last_undegraded":"2023-10-06T04:31:35.061339+0000","last_fullsized":"2023-10-06T04:31:35.061339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.27","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066488+0000","last_change":"2023-10-06T04:31:35.063285+0000","last_active":"2023-10-06T04:31:35.066488+0000","last_peered":"2023-10-06T04:31:35.066488+0000","last_clean":"2023-10-06T04:31:35.066488+0000","last_became_active":"2023-10-06T04:31:35.062975+0000","last_became_peered":"2023-10-06T04:31:35.062975+0000","last_unstale":"2023-10-06T04:31:35.066488+0000","last_undegraded":"2023-10-06T04:31:35.066488+0000","last_fullsized":"2023-10-06T04:31:35.066488+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.26","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063967+0000","last_change":"2023-10-06T04:31:35.058231+0000","last_active":"2023-10-06T04:31:35.063967+0000","last_peered":"2023-10-06T04:31:35.063967+0000","last_clean":"2023-10-06T04:31:35.063967+0000","last_became_active":"2023-10-06T04:31:35.057944+0000","last_became_peered":"2023-10-06T04:31:35.057944+0000","last_unstale":"2023-10-06T04:31:35.063967+0000","last_undegraded":"2023-10-06T04:31:35.063967+0000","last_fullsized":"2023-10-06T04:31:35.063967+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.25","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065515+0000","last_change":"2023-10-06T04:31:35.063120+0000","last_active":"2023-10-06T04:31:35.065515+0000","last_peered":"2023-10-06T04:31:35.065515+0000","last_clean":"2023-10-06T04:31:35.065515+0000","last_became_active":"2023-10-06T04:31:35.062868+0000","last_became_peered":"2023-10-06T04:31:35.062868+0000","last_unstale":"2023-10-06T04:31:35.065515+0000","last_undegraded":"2023-10-06T04:31:35.065515+0000","last_fullsized":"2023-10-06T04:31:35.065515+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.251995+0000","last_change":"2023-10-06T04:31:34.042620+0000","last_active":"2023-10-06T04:31:35.251995+0000","last_peered":"2023-10-06T04:31:35.251995+0000","last_clean":"2023-10-06T04:31:35.251995+0000","last_became_active":"2023-10-06T04:31:34.042023+0000","last_became_peered":"2023-10-06T04:31:34.042023+0000","last_unstale":"2023-10-06T04:31:35.251995+0000","last_undegraded":"2023-10-06T04:31:35.251995+0000","last_fullsized":"2023-10-06T04:31:35.251995+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066558+0000","last_change":"2023-10-06T04:31:35.060955+0000","last_active":"2023-10-06T04:31:35.066558+0000","last_peered":"2023-10-06T04:31:35.066558+0000","last_clean":"2023-10-06T04:31:35.066558+0000","last_became_active":"2023-10-06T04:31:35.060124+0000","last_became_peered":"2023-10-06T04:31:35.060124+0000","last_unstale":"2023-10-06T04:31:35.066558+0000","last_undegraded":"2023-10-06T04:31:35.066558+0000","last_fullsized":"2023-10-06T04:31:35.066558+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052353+0000","last_change":"2023-10-06T04:31:34.036811+0000","last_active":"2023-10-06T04:31:35.052353+0000","last_peered":"2023-10-06T04:31:35.052353+0000","last_clean":"2023-10-06T04:31:35.052353+0000","last_became_active":"2023-10-06T04:31:34.036513+0000","last_became_peered":"2023-10-06T04:31:34.036513+0000","last_unstale":"2023-10-06T04:31:35.052353+0000","last_undegraded":"2023-10-06T04:31:35.052353+0000","last_fullsized":"2023-10-06T04:31:35.052353+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062783+0000","last_change":"2023-10-06T04:31:35.055379+0000","last_active":"2023-10-06T04:31:35.062783+0000","last_peered":"2023-10-06T04:31:35.062783+0000","last_clean":"2023-10-06T04:31:35.062783+0000","last_became_active":"2023-10-06T04:31:35.055088+0000","last_became_peered":"2023-10-06T04:31:35.055088+0000","last_unstale":"2023-10-06T04:31:35.062783+0000","last_undegraded":"2023-10-06T04:31:35.062783+0000","last_fullsized":"2023-10-06T04:31:35.062783+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.048203+0000","last_change":"2023-10-06T04:31:34.054496+0000","last_active":"2023-10-06T04:31:35.048203+0000","last_peered":"2023-10-06T04:31:35.048203+0000","last_clean":"2023-10-06T04:31:35.048203+0000","last_became_active":"2023-10-06T04:31:34.054225+0000","last_became_peered":"2023-10-06T04:31:34.054225+0000","last_unstale":"2023-10-06T04:31:35.048203+0000","last_undegraded":"2023-10-06T04:31:35.048203+0000","last_fullsized":"2023-10-06T04:31:35.048203+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062762+0000","last_change":"2023-10-06T04:31:35.058805+0000","last_active":"2023-10-06T04:31:35.062762+0000","last_peered":"2023-10-06T04:31:35.062762+0000","last_clean":"2023-10-06T04:31:35.062762+0000","last_became_active":"2023-10-06T04:31:35.058524+0000","last_became_peered":"2023-10-06T04:31:35.058524+0000","last_unstale":"2023-10-06T04:31:35.062762+0000","last_undegraded":"2023-10-06T04:31:35.062762+0000","last_fullsized":"2023-10-06T04:31:35.062762+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046385+0000","last_change":"2023-10-06T04:31:34.042810+0000","last_active":"2023-10-06T04:31:35.046385+0000","last_peered":"2023-10-06T04:31:35.046385+0000","last_clean":"2023-10-06T04:31:35.046385+0000","last_became_active":"2023-10-06T04:31:34.040519+0000","last_became_peered":"2023-10-06T04:31:34.040519+0000","last_unstale":"2023-10-06T04:31:35.046385+0000","last_undegraded":"2023-10-06T04:31:35.046385+0000","last_fullsized":"2023-10-06T04:31:35.046385+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:57:52.489332+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066465+0000","last_change":"2023-10-06T04:31:35.061039+0000","last_active":"2023-10-06T04:31:35.066465+0000","last_peered":"2023-10-06T04:31:35.066465+0000","last_clean":"2023-10-06T04:31:35.066465+0000","last_became_active":"2023-10-06T04:31:35.060281+0000","last_became_peered":"2023-10-06T04:31:35.060281+0000","last_unstale":"2023-10-06T04:31:35.066465+0000","last_undegraded":"2023-10-06T04:31:35.066465+0000","last_fullsized":"2023-10-06T04:31:35.066465+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.053685+0000","last_change":"2023-10-06T04:31:34.039478+0000","last_active":"2023-10-06T04:31:35.053685+0000","last_peered":"2023-10-06T04:31:35.053685+0000","last_clean":"2023-10-06T04:31:35.053685+0000","last_became_active":"2023-10-06T04:31:34.039209+0000","last_became_peered":"2023-10-06T04:31:34.039209+0000","last_unstale":"2023-10-06T04:31:35.053685+0000","last_undegraded":"2023-10-06T04:31:35.053685+0000","last_fullsized":"2023-10-06T04:31:35.053685+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066712+0000","last_change":"2023-10-06T04:31:35.063717+0000","last_active":"2023-10-06T04:31:35.066712+0000","last_peered":"2023-10-06T04:31:35.066712+0000","last_clean":"2023-10-06T04:31:35.066712+0000","last_became_active":"2023-10-06T04:31:35.060241+0000","last_became_peered":"2023-10-06T04:31:35.060241+0000","last_unstale":"2023-10-06T04:31:35.066712+0000","last_undegraded":"2023-10-06T04:31:35.066712+0000","last_fullsized":"2023-10-06T04:31:35.066712+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:36:21.995977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.15","version":"22'3","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.254772+0000","last_change":"2023-10-06T04:31:34.045777+0000","last_active":"2023-10-06T04:31:35.254772+0000","last_peered":"2023-10-06T04:31:35.254772+0000","last_clean":"2023-10-06T04:31:35.254772+0000","last_became_active":"2023-10-06T04:31:34.045498+0000","last_became_peered":"2023-10-06T04:31:34.045498+0000","last_unstale":"2023-10-06T04:31:35.254772+0000","last_undegraded":"2023-10-06T04:31:35.254772+0000","last_fullsized":"2023-10-06T04:31:35.254772+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:15.139941+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066500+0000","last_change":"2023-10-06T04:31:35.063439+0000","last_active":"2023-10-06T04:31:35.066500+0000","last_peered":"2023-10-06T04:31:35.066500+0000","last_clean":"2023-10-06T04:31:35.066500+0000","last_became_active":"2023-10-06T04:31:35.063117+0000","last_became_peered":"2023-10-06T04:31:35.063117+0000","last_unstale":"2023-10-06T04:31:35.066500+0000","last_undegraded":"2023-10-06T04:31:35.066500+0000","last_fullsized":"2023-10-06T04:31:35.066500+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:45:16.144634+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"22'6","reported_seq":16,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.260205+0000","last_change":"2023-10-06T04:31:34.039091+0000","last_active":"2023-10-06T04:31:35.260205+0000","last_peered":"2023-10-06T04:31:35.260205+0000","last_clean":"2023-10-06T04:31:35.260205+0000","last_became_active":"2023-10-06T04:31:34.038778+0000","last_became_peered":"2023-10-06T04:31:34.038778+0000","last_unstale":"2023-10-06T04:31:35.260205+0000","last_undegraded":"2023-10-06T04:31:35.260205+0000","last_fullsized":"2023-10-06T04:31:35.260205+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.058518+0000","last_change":"2023-10-06T04:31:35.052851+0000","last_active":"2023-10-06T04:31:35.058518+0000","last_peered":"2023-10-06T04:31:35.058518+0000","last_clean":"2023-10-06T04:31:35.058518+0000","last_became_active":"2023-10-06T04:31:35.052560+0000","last_became_peered":"2023-10-06T04:31:35.052560+0000","last_unstale":"2023-10-06T04:31:35.058518+0000","last_undegraded":"2023-10-06T04:31:35.058518+0000","last_fullsized":"2023-10-06T04:31:35.058518+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:50:39.105270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.13","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255046+0000","last_change":"2023-10-06T04:31:34.046053+0000","last_active":"2023-10-06T04:31:35.255046+0000","last_peered":"2023-10-06T04:31:35.255046+0000","last_clean":"2023-10-06T04:31:35.255046+0000","last_became_active":"2023-10-06T04:31:34.045776+0000","last_became_peered":"2023-10-06T04:31:34.045776+0000","last_unstale":"2023-10-06T04:31:35.255046+0000","last_undegraded":"2023-10-06T04:31:35.255046+0000","last_fullsized":"2023-10-06T04:31:35.255046+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:20:40.537300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066602+0000","last_change":"2023-10-06T04:31:35.062504+0000","last_active":"2023-10-06T04:31:35.066602+0000","last_peered":"2023-10-06T04:31:35.066602+0000","last_clean":"2023-10-06T04:31:35.066602+0000","last_became_active":"2023-10-06T04:31:35.060884+0000","last_became_peered":"2023-10-06T04:31:35.060884+0000","last_unstale":"2023-10-06T04:31:35.066602+0000","last_undegraded":"2023-10-06T04:31:35.066602+0000","last_fullsized":"2023-10-06T04:31:35.066602+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.045014+0000","last_change":"2023-10-06T04:31:34.042234+0000","last_active":"2023-10-06T04:31:35.045014+0000","last_peered":"2023-10-06T04:31:35.045014+0000","last_clean":"2023-10-06T04:31:35.045014+0000","last_became_active":"2023-10-06T04:31:34.041959+0000","last_became_peered":"2023-10-06T04:31:34.041959+0000","last_unstale":"2023-10-06T04:31:35.045014+0000","last_undegraded":"2023-10-06T04:31:35.045014+0000","last_fullsized":"2023-10-06T04:31:35.045014+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064999+0000","last_change":"2023-10-06T04:31:35.061678+0000","last_active":"2023-10-06T04:31:35.064999+0000","last_peered":"2023-10-06T04:31:35.064999+0000","last_clean":"2023-10-06T04:31:35.064999+0000","last_became_active":"2023-10-06T04:31:35.061383+0000","last_became_peered":"2023-10-06T04:31:35.061383+0000","last_unstale":"2023-10-06T04:31:35.064999+0000","last_undegraded":"2023-10-06T04:31:35.064999+0000","last_fullsized":"2023-10-06T04:31:35.064999+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051432+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:35.051432+0000","last_peered":"2023-10-06T04:31:35.051432+0000","last_clean":"2023-10-06T04:31:35.051432+0000","last_became_active":"2023-10-06T04:31:34.046107+0000","last_became_peered":"2023-10-06T04:31:34.046107+0000","last_unstale":"2023-10-06T04:31:35.051432+0000","last_undegraded":"2023-10-06T04:31:35.051432+0000","last_fullsized":"2023-10-06T04:31:35.051432+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062761+0000","last_change":"2023-10-06T04:31:35.055168+0000","last_active":"2023-10-06T04:31:35.062761+0000","last_peered":"2023-10-06T04:31:35.062761+0000","last_clean":"2023-10-06T04:31:35.062761+0000","last_became_active":"2023-10-06T04:31:35.054903+0000","last_became_peered":"2023-10-06T04:31:35.054903+0000","last_unstale":"2023-10-06T04:31:35.062761+0000","last_undegraded":"2023-10-06T04:31:35.062761+0000","last_fullsized":"2023-10-06T04:31:35.062761+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.10","version":"22'8","reported_seq":18,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257527+0000","last_change":"2023-10-06T04:31:34.039233+0000","last_active":"2023-10-06T04:31:35.257527+0000","last_peered":"2023-10-06T04:31:35.257527+0000","last_clean":"2023-10-06T04:31:35.257527+0000","last_became_active":"2023-10-06T04:31:34.038916+0000","last_became_peered":"2023-10-06T04:31:34.038916+0000","last_unstale":"2023-10-06T04:31:35.257527+0000","last_undegraded":"2023-10-06T04:31:35.257527+0000","last_fullsized":"2023-10-06T04:31:35.257527+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:52:31.215430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066585+0000","last_change":"2023-10-06T04:31:35.062145+0000","last_active":"2023-10-06T04:31:35.066585+0000","last_peered":"2023-10-06T04:31:35.066585+0000","last_clean":"2023-10-06T04:31:35.066585+0000","last_became_active":"2023-10-06T04:31:35.061180+0000","last_became_peered":"2023-10-06T04:31:35.061180+0000","last_unstale":"2023-10-06T04:31:35.066585+0000","last_undegraded":"2023-10-06T04:31:35.066585+0000","last_fullsized":"2023-10-06T04:31:35.066585+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.f","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255042+0000","last_change":"2023-10-06T04:31:34.043099+0000","last_active":"2023-10-06T04:31:35.255042+0000","last_peered":"2023-10-06T04:31:35.255042+0000","last_clean":"2023-10-06T04:31:35.255042+0000","last_became_active":"2023-10-06T04:31:34.042869+0000","last_became_peered":"2023-10-06T04:31:34.042869+0000","last_unstale":"2023-10-06T04:31:35.255042+0000","last_undegraded":"2023-10-06T04:31:35.255042+0000","last_fullsized":"2023-10-06T04:31:35.255042+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065006+0000","last_change":"2023-10-06T04:31:35.061606+0000","last_active":"2023-10-06T04:31:35.065006+0000","last_peered":"2023-10-06T04:31:35.065006+0000","last_clean":"2023-10-06T04:31:35.065006+0000","last_became_active":"2023-10-06T04:31:35.061333+0000","last_became_peered":"2023-10-06T04:31:35.061333+0000","last_unstale":"2023-10-06T04:31:35.065006+0000","last_undegraded":"2023-10-06T04:31:35.065006+0000","last_fullsized":"2023-10-06T04:31:35.065006+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.e","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255162+0000","last_change":"2023-10-06T04:31:34.045981+0000","last_active":"2023-10-06T04:31:35.255162+0000","last_peered":"2023-10-06T04:31:35.255162+0000","last_clean":"2023-10-06T04:31:35.255162+0000","last_became_active":"2023-10-06T04:31:34.045683+0000","last_became_peered":"2023-10-06T04:31:34.045683+0000","last_unstale":"2023-10-06T04:31:35.255162+0000","last_undegraded":"2023-10-06T04:31:35.255162+0000","last_fullsized":"2023-10-06T04:31:35.255162+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:22.428758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062438+0000","last_change":"2023-10-06T04:31:35.059429+0000","last_active":"2023-10-06T04:31:35.062438+0000","last_peered":"2023-10-06T04:31:35.062438+0000","last_clean":"2023-10-06T04:31:35.062438+0000","last_became_active":"2023-10-06T04:31:35.058829+0000","last_became_peered":"2023-10-06T04:31:35.058829+0000","last_unstale":"2023-10-06T04:31:35.062438+0000","last_undegraded":"2023-10-06T04:31:35.062438+0000","last_fullsized":"2023-10-06T04:31:35.062438+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:21:02.736534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.d","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257596+0000","last_change":"2023-10-06T04:31:34.048038+0000","last_active":"2023-10-06T04:31:35.257596+0000","last_peered":"2023-10-06T04:31:35.257596+0000","last_clean":"2023-10-06T04:31:35.257596+0000","last_became_active":"2023-10-06T04:31:34.047780+0000","last_became_peered":"2023-10-06T04:31:34.047780+0000","last_unstale":"2023-10-06T04:31:35.257596+0000","last_undegraded":"2023-10-06T04:31:35.257596+0000","last_fullsized":"2023-10-06T04:31:35.257596+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:51:21.519366+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.060134+0000","last_change":"2023-10-06T04:31:35.057281+0000","last_active":"2023-10-06T04:31:35.060134+0000","last_peered":"2023-10-06T04:31:35.060134+0000","last_clean":"2023-10-06T04:31:35.060134+0000","last_became_active":"2023-10-06T04:31:35.056855+0000","last_became_peered":"2023-10-06T04:31:35.056855+0000","last_unstale":"2023-10-06T04:31:35.060134+0000","last_undegraded":"2023-10-06T04:31:35.060134+0000","last_fullsized":"2023-10-06T04:31:35.060134+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:23.765556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051180+0000","last_change":"2023-10-06T04:31:34.046008+0000","last_active":"2023-10-06T04:31:35.051180+0000","last_peered":"2023-10-06T04:31:35.051180+0000","last_clean":"2023-10-06T04:31:35.051180+0000","last_became_active":"2023-10-06T04:31:34.045732+0000","last_became_peered":"2023-10-06T04:31:34.045732+0000","last_unstale":"2023-10-06T04:31:35.051180+0000","last_undegraded":"2023-10-06T04:31:35.051180+0000","last_fullsized":"2023-10-06T04:31:35.051180+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:23:22.760862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063405+0000","last_change":"2023-10-06T04:31:35.059341+0000","last_active":"2023-10-06T04:31:35.063405+0000","last_peered":"2023-10-06T04:31:35.063405+0000","last_clean":"2023-10-06T04:31:35.063405+0000","last_became_active":"2023-10-06T04:31:35.059087+0000","last_became_peered":"2023-10-06T04:31:35.059087+0000","last_unstale":"2023-10-06T04:31:35.063405+0000","last_undegraded":"2023-10-06T04:31:35.063405+0000","last_fullsized":"2023-10-06T04:31:35.063405+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.b","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.249472+0000","last_change":"2023-10-06T04:31:34.045304+0000","last_active":"2023-10-06T04:31:35.249472+0000","last_peered":"2023-10-06T04:31:35.249472+0000","last_clean":"2023-10-06T04:31:35.249472+0000","last_became_active":"2023-10-06T04:31:34.044891+0000","last_became_peered":"2023-10-06T04:31:34.044891+0000","last_unstale":"2023-10-06T04:31:35.249472+0000","last_undegraded":"2023-10-06T04:31:35.249472+0000","last_fullsized":"2023-10-06T04:31:35.249472+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059237+0000","last_change":"2023-10-06T04:31:35.055035+0000","last_active":"2023-10-06T04:31:35.059237+0000","last_peered":"2023-10-06T04:31:35.059237+0000","last_clean":"2023-10-06T04:31:35.059237+0000","last_became_active":"2023-10-06T04:31:35.054727+0000","last_became_peered":"2023-10-06T04:31:35.054727+0000","last_unstale":"2023-10-06T04:31:35.059237+0000","last_undegraded":"2023-10-06T04:31:35.059237+0000","last_fullsized":"2023-10-06T04:31:35.059237+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:02:31.805865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.043967+0000","last_change":"2023-10-06T04:31:34.041723+0000","last_active":"2023-10-06T04:31:35.043967+0000","last_peered":"2023-10-06T04:31:35.043967+0000","last_clean":"2023-10-06T04:31:35.043967+0000","last_became_active":"2023-10-06T04:31:34.041386+0000","last_became_peered":"2023-10-06T04:31:34.041386+0000","last_unstale":"2023-10-06T04:31:35.043967+0000","last_undegraded":"2023-10-06T04:31:35.043967+0000","last_fullsized":"2023-10-06T04:31:35.043967+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:09:58.993061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059756+0000","last_change":"2023-10-06T04:31:35.057831+0000","last_active":"2023-10-06T04:31:35.059756+0000","last_peered":"2023-10-06T04:31:35.059756+0000","last_clean":"2023-10-06T04:31:35.059756+0000","last_became_active":"2023-10-06T04:31:35.057257+0000","last_became_peered":"2023-10-06T04:31:35.057257+0000","last_unstale":"2023-10-06T04:31:35.059756+0000","last_undegraded":"2023-10-06T04:31:35.059756+0000","last_fullsized":"2023-10-06T04:31:35.059756+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046478+0000","last_change":"2023-10-06T04:31:34.035016+0000","last_active":"2023-10-06T04:31:35.046478+0000","last_peered":"2023-10-06T04:31:35.046478+0000","last_clean":"2023-10-06T04:31:35.046478+0000","last_became_active":"2023-10-06T04:31:34.034721+0000","last_became_peered":"2023-10-06T04:31:34.034721+0000","last_unstale":"2023-10-06T04:31:35.046478+0000","last_undegraded":"2023-10-06T04:31:35.046478+0000","last_fullsized":"2023-10-06T04:31:35.046478+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.051940+0000","last_change":"2023-10-06T04:31:35.049984+0000","last_active":"2023-10-06T04:31:35.051940+0000","last_peered":"2023-10-06T04:31:35.051940+0000","last_clean":"2023-10-06T04:31:35.051940+0000","last_became_active":"2023-10-06T04:31:35.049687+0000","last_became_peered":"2023-10-06T04:31:35.049687+0000","last_unstale":"2023-10-06T04:31:35.051940+0000","last_undegraded":"2023-10-06T04:31:35.051940+0000","last_fullsized":"2023-10-06T04:31:35.051940+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.048258+0000","last_change":"2023-10-06T04:31:34.046626+0000","last_active":"2023-10-06T04:31:35.048258+0000","last_peered":"2023-10-06T04:31:35.048258+0000","last_clean":"2023-10-06T04:31:35.048258+0000","last_became_active":"2023-10-06T04:31:34.044656+0000","last_became_peered":"2023-10-06T04:31:34.044656+0000","last_unstale":"2023-10-06T04:31:35.048258+0000","last_undegraded":"2023-10-06T04:31:35.048258+0000","last_fullsized":"2023-10-06T04:31:35.048258+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064326+0000","last_change":"2023-10-06T04:31:35.059496+0000","last_active":"2023-10-06T04:31:35.064326+0000","last_peered":"2023-10-06T04:31:35.064326+0000","last_clean":"2023-10-06T04:31:35.064326+0000","last_became_active":"2023-10-06T04:31:35.059187+0000","last_became_peered":"2023-10-06T04:31:35.059187+0000","last_unstale":"2023-10-06T04:31:35.064326+0000","last_undegraded":"2023-10-06T04:31:35.064326+0000","last_fullsized":"2023-10-06T04:31:35.064326+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:17:39.911971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052813+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:31:35.052813+0000","last_peered":"2023-10-06T04:31:35.052813+0000","last_clean":"2023-10-06T04:31:35.052813+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:31:35.052813+0000","last_undegraded":"2023-10-06T04:31:35.052813+0000","last_fullsized":"2023-10-06T04:31:35.052813+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.057242+0000","last_change":"2023-10-06T04:31:34.036361+0000","last_active":"2023-10-06T04:31:35.057242+0000","last_peered":"2023-10-06T04:31:35.057242+0000","last_clean":"2023-10-06T04:31:35.057242+0000","last_became_active":"2023-10-06T04:31:34.036048+0000","last_became_peered":"2023-10-06T04:31:34.036048+0000","last_unstale":"2023-10-06T04:31:35.057242+0000","last_undegraded":"2023-10-06T04:31:35.057242+0000","last_fullsized":"2023-10-06T04:31:35.057242+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:07.164615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065390+0000","last_change":"2023-10-06T04:31:35.061294+0000","last_active":"2023-10-06T04:31:35.065390+0000","last_peered":"2023-10-06T04:31:35.065390+0000","last_clean":"2023-10-06T04:31:35.065390+0000","last_became_active":"2023-10-06T04:31:35.061034+0000","last_became_peered":"2023-10-06T04:31:35.061034+0000","last_unstale":"2023-10-06T04:31:35.065390+0000","last_undegraded":"2023-10-06T04:31:35.065390+0000","last_fullsized":"2023-10-06T04:31:35.065390+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:59:59.355740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055363+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:31:35.055363+0000","last_peered":"2023-10-06T04:31:35.055363+0000","last_clean":"2023-10-06T04:31:35.055363+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:31:35.055363+0000","last_undegraded":"2023-10-06T04:31:35.055363+0000","last_fullsized":"2023-10-06T04:31:35.055363+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.046081+0000","last_change":"2023-10-06T04:31:34.043758+0000","last_active":"2023-10-06T04:31:35.046081+0000","last_peered":"2023-10-06T04:31:35.046081+0000","last_clean":"2023-10-06T04:31:35.046081+0000","last_became_active":"2023-10-06T04:31:34.043386+0000","last_became_peered":"2023-10-06T04:31:34.043386+0000","last_unstale":"2023-10-06T04:31:35.046081+0000","last_undegraded":"2023-10-06T04:31:35.046081+0000","last_fullsized":"2023-10-06T04:31:35.046081+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064019+0000","last_change":"2023-10-06T04:31:35.058248+0000","last_active":"2023-10-06T04:31:35.064019+0000","last_peered":"2023-10-06T04:31:35.064019+0000","last_clean":"2023-10-06T04:31:35.064019+0000","last_became_active":"2023-10-06T04:31:35.057955+0000","last_became_peered":"2023-10-06T04:31:35.057955+0000","last_unstale":"2023-10-06T04:31:35.064019+0000","last_undegraded":"2023-10-06T04:31:35.064019+0000","last_fullsized":"2023-10-06T04:31:35.064019+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:27:36.038264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.040377+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:31:35.040377+0000","last_peered":"2023-10-06T04:31:35.040377+0000","last_clean":"2023-10-06T04:31:35.040377+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:31:35.040377+0000","last_undegraded":"2023-10-06T04:31:35.040377+0000","last_fullsized":"2023-10-06T04:31:35.040377+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052376+0000","last_change":"2023-10-06T04:31:34.041900+0000","last_active":"2023-10-06T04:31:35.052376+0000","last_peered":"2023-10-06T04:31:35.052376+0000","last_clean":"2023-10-06T04:31:35.052376+0000","last_became_active":"2023-10-06T04:31:34.041621+0000","last_became_peered":"2023-10-06T04:31:34.041621+0000","last_unstale":"2023-10-06T04:31:35.052376+0000","last_undegraded":"2023-10-06T04:31:35.052376+0000","last_fullsized":"2023-10-06T04:31:35.052376+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.064339+0000","last_change":"2023-10-06T04:31:35.060634+0000","last_active":"2023-10-06T04:31:35.064339+0000","last_peered":"2023-10-06T04:31:35.064339+0000","last_clean":"2023-10-06T04:31:35.064339+0000","last_became_active":"2023-10-06T04:31:35.060183+0000","last_became_peered":"2023-10-06T04:31:35.060183+0000","last_unstale":"2023-10-06T04:31:35.064339+0000","last_undegraded":"2023-10-06T04:31:35.064339+0000","last_fullsized":"2023-10-06T04:31:35.064339+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:45:48.971463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.055245+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:31:35.055245+0000","last_peered":"2023-10-06T04:31:35.055245+0000","last_clean":"2023-10-06T04:31:35.055245+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:31:35.055245+0000","last_undegraded":"2023-10-06T04:31:35.055245+0000","last_fullsized":"2023-10-06T04:31:35.055245+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":166,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049727+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:31:35.049727+0000","last_peered":"2023-10-06T04:31:35.049727+0000","last_clean":"2023-10-06T04:31:35.049727+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:31:35.049727+0000","last_undegraded":"2023-10-06T04:31:35.049727+0000","last_fullsized":"2023-10-06T04:31:35.049727+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.048092+0000","last_change":"2023-10-06T04:31:34.039806+0000","last_active":"2023-10-06T04:31:35.048092+0000","last_peered":"2023-10-06T04:31:35.048092+0000","last_clean":"2023-10-06T04:31:35.048092+0000","last_became_active":"2023-10-06T04:31:34.039496+0000","last_became_peered":"2023-10-06T04:31:34.039496+0000","last_unstale":"2023-10-06T04:31:35.048092+0000","last_undegraded":"2023-10-06T04:31:35.048092+0000","last_fullsized":"2023-10-06T04:31:35.048092+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052446+0000","last_change":"2023-10-06T04:31:35.050727+0000","last_active":"2023-10-06T04:31:35.052446+0000","last_peered":"2023-10-06T04:31:35.052446+0000","last_clean":"2023-10-06T04:31:35.052446+0000","last_became_active":"2023-10-06T04:31:35.050446+0000","last_became_peered":"2023-10-06T04:31:35.050446+0000","last_unstale":"2023-10-06T04:31:35.052446+0000","last_undegraded":"2023-10-06T04:31:35.052446+0000","last_fullsized":"2023-10-06T04:31:35.052446+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:28:08.169309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.050191+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:31:35.050191+0000","last_peered":"2023-10-06T04:31:35.050191+0000","last_clean":"2023-10-06T04:31:35.050191+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:31:35.050191+0000","last_undegraded":"2023-10-06T04:31:35.050191+0000","last_fullsized":"2023-10-06T04:31:35.050191+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.044071+0000","last_change":"2023-10-06T04:31:34.039113+0000","last_active":"2023-10-06T04:31:35.044071+0000","last_peered":"2023-10-06T04:31:35.044071+0000","last_clean":"2023-10-06T04:31:35.044071+0000","last_became_active":"2023-10-06T04:31:34.038410+0000","last_became_peered":"2023-10-06T04:31:34.038410+0000","last_unstale":"2023-10-06T04:31:35.044071+0000","last_undegraded":"2023-10-06T04:31:35.044071+0000","last_fullsized":"2023-10-06T04:31:35.044071+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:04:48.335745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063675+0000","last_change":"2023-10-06T04:31:35.059901+0000","last_active":"2023-10-06T04:31:35.063675+0000","last_peered":"2023-10-06T04:31:35.063675+0000","last_clean":"2023-10-06T04:31:35.063675+0000","last_became_active":"2023-10-06T04:31:35.059461+0000","last_became_peered":"2023-10-06T04:31:35.059461+0000","last_unstale":"2023-10-06T04:31:35.063675+0000","last_undegraded":"2023-10-06T04:31:35.063675+0000","last_fullsized":"2023-10-06T04:31:35.063675+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.049407+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:31:35.049407+0000","last_peered":"2023-10-06T04:31:35.049407+0000","last_clean":"2023-10-06T04:31:35.049407+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:31:35.049407+0000","last_undegraded":"2023-10-06T04:31:35.049407+0000","last_fullsized":"2023-10-06T04:31:35.049407+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052921+0000","last_change":"2023-10-06T04:31:34.042477+0000","last_active":"2023-10-06T04:31:35.052921+0000","last_peered":"2023-10-06T04:31:35.052921+0000","last_clean":"2023-10-06T04:31:35.052921+0000","last_became_active":"2023-10-06T04:31:34.041582+0000","last_became_peered":"2023-10-06T04:31:34.041582+0000","last_unstale":"2023-10-06T04:31:35.052921+0000","last_undegraded":"2023-10-06T04:31:35.052921+0000","last_fullsized":"2023-10-06T04:31:35.052921+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:24:55.181691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063792+0000","last_change":"2023-10-06T04:31:35.061904+0000","last_active":"2023-10-06T04:31:35.063792+0000","last_peered":"2023-10-06T04:31:35.063792+0000","last_clean":"2023-10-06T04:31:35.063792+0000","last_became_active":"2023-10-06T04:31:35.061655+0000","last_became_peered":"2023-10-06T04:31:35.061655+0000","last_unstale":"2023-10-06T04:31:35.063792+0000","last_undegraded":"2023-10-06T04:31:35.063792+0000","last_fullsized":"2023-10-06T04:31:35.063792+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:52.373580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.7","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255298+0000","last_change":"2023-10-06T04:31:34.042617+0000","last_active":"2023-10-06T04:31:35.255298+0000","last_peered":"2023-10-06T04:31:35.255298+0000","last_clean":"2023-10-06T04:31:35.255298+0000","last_became_active":"2023-10-06T04:31:34.042346+0000","last_became_peered":"2023-10-06T04:31:34.042346+0000","last_unstale":"2023-10-06T04:31:35.255298+0000","last_undegraded":"2023-10-06T04:31:35.255298+0000","last_fullsized":"2023-10-06T04:31:35.255298+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:27:39.821308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.052516+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:31:35.052516+0000","last_peered":"2023-10-06T04:31:35.052516+0000","last_clean":"2023-10-06T04:31:35.052516+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:31:35.052516+0000","last_undegraded":"2023-10-06T04:31:35.052516+0000","last_fullsized":"2023-10-06T04:31:35.052516+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066441+0000","last_change":"2023-10-06T04:31:35.060668+0000","last_active":"2023-10-06T04:31:35.066441+0000","last_peered":"2023-10-06T04:31:35.066441+0000","last_clean":"2023-10-06T04:31:35.066441+0000","last_became_active":"2023-10-06T04:31:35.059927+0000","last_became_peered":"2023-10-06T04:31:35.059927+0000","last_unstale":"2023-10-06T04:31:35.066441+0000","last_undegraded":"2023-10-06T04:31:35.066441+0000","last_fullsized":"2023-10-06T04:31:35.066441+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:42.071674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":13,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.043872+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:31:35.043872+0000","last_peered":"2023-10-06T04:31:35.043872+0000","last_clean":"2023-10-06T04:31:35.043872+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:31:35.043872+0000","last_undegraded":"2023-10-06T04:31:35.043872+0000","last_fullsized":"2023-10-06T04:31:35.043872+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"23'1","reported_seq":11,"reported_epoch":23,"state":"active+clean","last_fresh":"2023-10-06T04:31:37.159225+0000","last_change":"2023-10-06T04:31:34.044177+0000","last_active":"2023-10-06T04:31:37.159225+0000","last_peered":"2023-10-06T04:31:37.159225+0000","last_clean":"2023-10-06T04:31:37.159225+0000","last_became_active":"2023-10-06T04:31:34.043921+0000","last_became_peered":"2023-10-06T04:31:34.043921+0000","last_unstale":"2023-10-06T04:31:37.159225+0000","last_undegraded":"2023-10-06T04:31:37.159225+0000","last_fullsized":"2023-10-06T04:31:37.159225+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:09:32.321314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062416+0000","last_change":"2023-10-06T04:31:35.059320+0000","last_active":"2023-10-06T04:31:35.062416+0000","last_peered":"2023-10-06T04:31:35.062416+0000","last_clean":"2023-10-06T04:31:35.062416+0000","last_became_active":"2023-10-06T04:31:35.058458+0000","last_became_peered":"2023-10-06T04:31:35.058458+0000","last_unstale":"2023-10-06T04:31:35.062416+0000","last_undegraded":"2023-10-06T04:31:35.062416+0000","last_fullsized":"2023-10-06T04:31:35.062416+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:58:06.614824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1b","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.252891+0000","last_change":"2023-10-06T04:31:34.043203+0000","last_active":"2023-10-06T04:31:35.252891+0000","last_peered":"2023-10-06T04:31:35.252891+0000","last_clean":"2023-10-06T04:31:35.252891+0000","last_became_active":"2023-10-06T04:31:34.042886+0000","last_became_peered":"2023-10-06T04:31:34.042886+0000","last_unstale":"2023-10-06T04:31:35.252891+0000","last_undegraded":"2023-10-06T04:31:35.252891+0000","last_fullsized":"2023-10-06T04:31:35.252891+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:16:41.433643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066541+0000","last_change":"2023-10-06T04:31:35.057021+0000","last_active":"2023-10-06T04:31:35.066541+0000","last_peered":"2023-10-06T04:31:35.066541+0000","last_clean":"2023-10-06T04:31:35.066541+0000","last_became_active":"2023-10-06T04:31:35.056770+0000","last_became_peered":"2023-10-06T04:31:35.056770+0000","last_unstale":"2023-10-06T04:31:35.066541+0000","last_undegraded":"2023-10-06T04:31:35.066541+0000","last_fullsized":"2023-10-06T04:31:35.066541+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:05.320520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.1c","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.255172+0000","last_change":"2023-10-06T04:31:34.044080+0000","last_active":"2023-10-06T04:31:35.255172+0000","last_peered":"2023-10-06T04:31:35.255172+0000","last_clean":"2023-10-06T04:31:35.255172+0000","last_became_active":"2023-10-06T04:31:34.042225+0000","last_became_peered":"2023-10-06T04:31:34.042225+0000","last_unstale":"2023-10-06T04:31:35.255172+0000","last_undegraded":"2023-10-06T04:31:35.255172+0000","last_fullsized":"2023-10-06T04:31:35.255172+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:51:17.059659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.059100+0000","last_change":"2023-10-06T04:31:35.055021+0000","last_active":"2023-10-06T04:31:35.059100+0000","last_peered":"2023-10-06T04:31:35.059100+0000","last_clean":"2023-10-06T04:31:35.059100+0000","last_became_active":"2023-10-06T04:31:35.054737+0000","last_became_peered":"2023-10-06T04:31:35.054737+0000","last_unstale":"2023-10-06T04:31:35.059100+0000","last_undegraded":"2023-10-06T04:31:35.059100+0000","last_fullsized":"2023-10-06T04:31:35.059100+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:24:26.087138+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1d","version":"22'2","reported_seq":12,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257301+0000","last_change":"2023-10-06T04:31:34.046400+0000","last_active":"2023-10-06T04:31:35.257301+0000","last_peered":"2023-10-06T04:31:35.257301+0000","last_clean":"2023-10-06T04:31:35.257301+0000","last_became_active":"2023-10-06T04:31:34.041996+0000","last_became_peered":"2023-10-06T04:31:34.041996+0000","last_unstale":"2023-10-06T04:31:35.257301+0000","last_undegraded":"2023-10-06T04:31:35.257301+0000","last_fullsized":"2023-10-06T04:31:35.257301+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.062379+0000","last_change":"2023-10-06T04:31:35.059718+0000","last_active":"2023-10-06T04:31:35.062379+0000","last_peered":"2023-10-06T04:31:35.062379+0000","last_clean":"2023-10-06T04:31:35.062379+0000","last_became_active":"2023-10-06T04:31:35.059438+0000","last_became_peered":"2023-10-06T04:31:35.059438+0000","last_unstale":"2023-10-06T04:31:35.062379+0000","last_undegraded":"2023-10-06T04:31:35.062379+0000","last_fullsized":"2023-10-06T04:31:35.062379+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:22:25.514246+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.1e","version":"22'1","reported_seq":11,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.252417+0000","last_change":"2023-10-06T04:31:34.047689+0000","last_active":"2023-10-06T04:31:35.252417+0000","last_peered":"2023-10-06T04:31:35.252417+0000","last_clean":"2023-10-06T04:31:35.252417+0000","last_became_active":"2023-10-06T04:31:34.047311+0000","last_became_peered":"2023-10-06T04:31:34.047311+0000","last_unstale":"2023-10-06T04:31:35.252417+0000","last_undegraded":"2023-10-06T04:31:35.252417+0000","last_fullsized":"2023-10-06T04:31:35.252417+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:43:04.315826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.063861+0000","last_change":"2023-10-06T04:31:35.055295+0000","last_active":"2023-10-06T04:31:35.063861+0000","last_peered":"2023-10-06T04:31:35.063861+0000","last_clean":"2023-10-06T04:31:35.063861+0000","last_became_active":"2023-10-06T04:31:35.055019+0000","last_became_peered":"2023-10-06T04:31:35.055019+0000","last_unstale":"2023-10-06T04:31:35.063861+0000","last_undegraded":"2023-10-06T04:31:35.063861+0000","last_fullsized":"2023-10-06T04:31:35.063861+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"22'4","reported_seq":14,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.257663+0000","last_change":"2023-10-06T04:31:34.040784+0000","last_active":"2023-10-06T04:31:35.257663+0000","last_peered":"2023-10-06T04:31:35.257663+0000","last_clean":"2023-10-06T04:31:35.257663+0000","last_became_active":"2023-10-06T04:31:34.040127+0000","last_became_peered":"2023-10-06T04:31:34.040127+0000","last_unstale":"2023-10-06T04:31:35.257663+0000","last_undegraded":"2023-10-06T04:31:35.257663+0000","last_fullsized":"2023-10-06T04:31:35.257663+0000","mapping_epoch":20,"log_start":"0'0","ondisk_log_start":"0'0","created":20,"last_epoch_clean":21,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:33.017017+0000","last_clean_scrub_stamp":"2023-10-06T04:31:33.017017+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:17.300987+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054184+0000","last_change":"2023-10-06T04:31:35.052827+0000","last_active":"2023-10-06T04:31:35.054184+0000","last_peered":"2023-10-06T04:31:35.054184+0000","last_clean":"2023-10-06T04:31:35.054184+0000","last_became_active":"2023-10-06T04:31:35.052539+0000","last_became_peered":"2023-10-06T04:31:35.052539+0000","last_unstale":"2023-10-06T04:31:35.054184+0000","last_undegraded":"2023-10-06T04:31:35.054184+0000","last_fullsized":"2023-10-06T04:31:35.054184+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:22:22.039507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.20","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054464+0000","last_change":"2023-10-06T04:31:35.051004+0000","last_active":"2023-10-06T04:31:35.054464+0000","last_peered":"2023-10-06T04:31:35.054464+0000","last_clean":"2023-10-06T04:31:35.054464+0000","last_became_active":"2023-10-06T04:31:35.050694+0000","last_became_peered":"2023-10-06T04:31:35.050694+0000","last_unstale":"2023-10-06T04:31:35.054464+0000","last_undegraded":"2023-10-06T04:31:35.054464+0000","last_fullsized":"2023-10-06T04:31:35.054464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:58:18.305681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.21","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066625+0000","last_change":"2023-10-06T04:31:35.062748+0000","last_active":"2023-10-06T04:31:35.066625+0000","last_peered":"2023-10-06T04:31:35.066625+0000","last_clean":"2023-10-06T04:31:35.066625+0000","last_became_active":"2023-10-06T04:31:35.060650+0000","last_became_peered":"2023-10-06T04:31:35.060650+0000","last_unstale":"2023-10-06T04:31:35.066625+0000","last_undegraded":"2023-10-06T04:31:35.066625+0000","last_fullsized":"2023-10-06T04:31:35.066625+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:20.946258+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.22","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.054198+0000","last_change":"2023-10-06T04:31:35.052675+0000","last_active":"2023-10-06T04:31:35.054198+0000","last_peered":"2023-10-06T04:31:35.054198+0000","last_clean":"2023-10-06T04:31:35.054198+0000","last_became_active":"2023-10-06T04:31:35.052379+0000","last_became_peered":"2023-10-06T04:31:35.052379+0000","last_unstale":"2023-10-06T04:31:35.054198+0000","last_undegraded":"2023-10-06T04:31:35.054198+0000","last_fullsized":"2023-10-06T04:31:35.054198+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:36:40.270087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.23","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.066464+0000","last_change":"2023-10-06T04:31:35.061097+0000","last_active":"2023-10-06T04:31:35.066464+0000","last_peered":"2023-10-06T04:31:35.066464+0000","last_clean":"2023-10-06T04:31:35.066464+0000","last_became_active":"2023-10-06T04:31:35.060817+0000","last_became_peered":"2023-10-06T04:31:35.060817+0000","last_unstale":"2023-10-06T04:31:35.066464+0000","last_undegraded":"2023-10-06T04:31:35.066464+0000","last_fullsized":"2023-10-06T04:31:35.066464+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:26:18.474107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.24","version":"0'0","reported_seq":9,"reported_epoch":22,"state":"active+clean","last_fresh":"2023-10-06T04:31:35.065474+0000","last_change":"2023-10-06T04:31:35.062003+0000","last_active":"2023-10-06T04:31:35.065474+0000","last_peered":"2023-10-06T04:31:35.065474+0000","last_clean":"2023-10-06T04:31:35.065474+0000","last_became_active":"2023-10-06T04:31:35.061553+0000","last_became_peered":"2023-10-06T04:31:35.061553+0000","last_unstale":"2023-10-06T04:31:35.065474+0000","last_undegraded":"2023-10-06T04:31:35.065474+0000","last_fullsized":"2023-10-06T04:31:35.065474+0000","mapping_epoch":21,"log_start":"0'0","ondisk_log_start":"0'0","created":21,"last_epoch_clean":22,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:34.021711+0000","last_clean_scrub_stamp":"2023-10-06T04:31:34.021711+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:12:56.874669+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":3,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574856,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298132,"kb_used_data":844,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073708,"statfs":{"total":96636764160,"available":96331476992,"internally_reserved":0,"allocated":864256,"data_stored":681439,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.70399999999999996}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.68700000000000006}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65800000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76700000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74399999999999999}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42299999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79500000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574855,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.35299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48399999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58399999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.495}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.308}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52800000000000002}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.497}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47599999999999998}]}]},{"osd":0,"up_from":13,"seq":55834574856,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298108,"kb_used_data":820,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073732,"statfs":{"total":96636764160,"available":96331501568,"internally_reserved":0,"allocated":839680,"data_stored":680241,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.633}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.7150000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.345}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.413}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.6899999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.468}]}]},{"osd":2,"up_from":13,"seq":55834574856,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297524,"kb_used_data":236,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074316,"statfs":{"total":96636764160,"available":96332099584,"internally_reserved":0,"allocated":241664,"data_stored":89319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.27000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.371}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.505}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63200000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574856,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297528,"kb_used_data":240,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074312,"statfs":{"total":96636764160,"available":96332095488,"internally_reserved":0,"allocated":245760,"data_stored":89409,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.377}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.29999999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.33800000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574857,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297544,"kb_used_data":256,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074296,"statfs":{"total":96636764160,"available":96332079104,"internally_reserved":0,"allocated":262144,"data_stored":90911,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52900000000000003}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.437}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51000000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46200000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57099999999999995}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54900000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]}]},{"osd":5,"up_from":13,"seq":55834574856,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297532,"kb_used_data":244,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074308,"statfs":{"total":96636764160,"available":96332091392,"internally_reserved":0,"allocated":249856,"data_stored":89943,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54400000000000004}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60899999999999999}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47299999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60099999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1752,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:31:45.346 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T04:31:45.347 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-10-06T04:31:45.347 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-10-06T04:31:45.347 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-10-06T04:31:45.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.496+0000 7fc1e85a8700 1 Processor -- start 2023-10-06T04:31:45.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.496+0000 7fc1e85a8700 1 -- start start 2023-10-06T04:31:45.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e85a8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 0x7fc1e0157da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e85a8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e0157910 con 0x7fc1e006cd60 2023-10-06T04:31:45.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e85a8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1e006d240 con 0x7fc1e0156f60 2023-10-06T04:31:45.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e85a8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e006d520 con 0x7fc1e00ade40 2023-10-06T04:31:45.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5b43700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc1e006cd60 0x7fc1e00ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57032/0 (socket says 172.21.15.123:57032) 2023-10-06T04:31:45.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5b43700 1 -- 172.21.15.123:0/2586348708 learned_addr learned my addr 172.21.15.123:0/2586348708 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:45.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e6b45700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 0x7fc1e0157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006303074 0 0) 0x7fc1e0157910 con 0x7fc1e006cd60 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1cc003570 con 0x7fc1e006cd60 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1297938183 0 0) 0x7fc1e006d240 con 0x7fc1e0156f60 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc1e0157910 con 0x7fc1e0156f60 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3257293195 0 0) 0x7fc1cc003570 con 0x7fc1e006cd60 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 msgr2=0x7fc1e0157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 0x7fc1e0157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 >> v1:172.21.15.177:6789/0 conn(0x7fc1e0156f60 legacy=0x7fc1e015ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e006d800 con 0x7fc1e006cd60 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fc1d0001470 con 0x7fc1e006cd60 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 >> v1:172.21.15.123:6789/0 conn(0x7fc1e006cd60 legacy=0x7fc1e00ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc003c20 0x7fc1cc004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.497+0000 7fc1e5342700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1cc004810 0x7fc1cc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5342700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 0x7fc1cc0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e006d520 con 0x7fc1cc004810 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc00bb40 con 0x7fc1cc003c20 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1cc00bd00 con 0x7fc1e00ade40 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5b43700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1cc004810 0x7fc1cc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e6344700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc003c20 0x7fc1cc004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e6b45700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 0x7fc1cc0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5b43700 1 -- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 msgr2=0x7fc1cc0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5b43700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 0x7fc1cc0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5b43700 1 -- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc003c20 msgr2=0x7fc1cc004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5b43700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc003c20 0x7fc1cc004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5b43700 1 -- 172.21.15.123:0/2586348708 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1cc003570 con 0x7fc1cc004810 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e6b45700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e00ade40 0x7fc1cc0051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5b43700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1cc004810 0x7fc1cc004c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c2a0 tx=0x7fc1d0001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5c4b51f938e30d5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d000cb70 con 0x7fc1cc004810 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d000ccd0 con 0x7fc1cc004810 2023-10-06T04:31:45.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.498+0000 7fc1e5342700 1 -- 172.21.15.123:0/2586348708 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d0005b70 con 0x7fc1cc004810 2023-10-06T04:31:45.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.499+0000 7fc1e85a8700 1 -- 172.21.15.123:0/2586348708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1cc004810 msgr2=0x7fc1cc004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.499+0000 7fc1e85a8700 1 --2- 172.21.15.123:0/2586348708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1cc004810 0x7fc1cc004c80 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fc1d000c2a0 tx=0x7fc1d0001e10 comp rx=0 tx=0).stop 2023-10-06T04:31:45.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.499+0000 7fc1e6b45700 1 -- 172.21.15.123:0/2586348708 reap_dead start 2023-10-06T04:31:45.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.499+0000 7fc1e85a8700 1 -- 172.21.15.123:0/2586348708 shutdown_connections 2023-10-06T04:31:45.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.499+0000 7fc1e85a8700 1 -- 172.21.15.123:0/2586348708 >> 172.21.15.123:0/2586348708 conn(0x7fc1e00ce0c0 msgr2=0x7fc1e00ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:45.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.499+0000 7fc1e85a8700 1 -- 172.21.15.123:0/2586348708 shutdown_connections 2023-10-06T04:31:45.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.499+0000 7fc1e85a8700 1 -- 172.21.15.123:0/2586348708 wait complete. 2023-10-06T04:31:45.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.500+0000 7fc1e85a8700 1 Processor -- start 2023-10-06T04:31:45.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.500+0000 7fc1e85a8700 1 -- start start 2023-10-06T04:31:45.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.500+0000 7fc1e85a8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc004810 0x7fc1e0168cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.500+0000 7fc1e85a8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e0154b20 0x7fc1e00c9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.500+0000 7fc1e85a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e00c2c20 0x7fc1e0159de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.500+0000 7fc1e85a8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00ca3a0 con 0x7fc1e00c2c20 2023-10-06T04:31:45.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e85a8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00c2790 con 0x7fc1cc004810 2023-10-06T04:31:45.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e85a8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1e00c1b80 con 0x7fc1e0154b20 2023-10-06T04:31:45.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e00c2c20 0x7fc1e0159de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e00c2c20 0x7fc1e0159de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43498/0 (socket says 172.21.15.123:43498) 2023-10-06T04:31:45.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 -- 172.21.15.123:0/1133464767 learned_addr learned my addr 172.21.15.123:0/1133464767 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:45.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6344700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc004810 0x7fc1e0168cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e5b43700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e0154b20 0x7fc1e00c9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 -- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e0154b20 msgr2=0x7fc1e00c9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e0154b20 0x7fc1e00c9e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 -- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc004810 msgr2=0x7fc1e0168cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc004810 0x7fc1e0168cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 -- 172.21.15.123:0/1133464767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1e00c1e60 con 0x7fc1e00c2c20 2023-10-06T04:31:45.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e6b45700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e00c2c20 0x7fc1e0159de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00ca590 tx=0x7fc1d800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1d67fc700 1 -- 172.21.15.123:0/1133464767 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d800e050 con 0x7fc1e00c2c20 2023-10-06T04:31:45.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1d67fc700 1 -- 172.21.15.123:0/1133464767 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1d800f040 con 0x7fc1e00c2c20 2023-10-06T04:31:45.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1d67fc700 1 -- 172.21.15.123:0/1133464767 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1d8010df0 con 0x7fc1e00c2c20 2023-10-06T04:31:45.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1e00c2140 con 0x7fc1e00c2c20 2023-10-06T04:31:45.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.501+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1e0154f90 con 0x7fc1e00c2c20 2023-10-06T04:31:45.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.502+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1e006e690 con 0x7fc1e00c2c20 2023-10-06T04:31:45.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.502+0000 7fc1d67fc700 1 -- 172.21.15.123:0/1133464767 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc1d8003020 con 0x7fc1e00c2c20 2023-10-06T04:31:45.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.503+0000 7fc1d67fc700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc1c4066f30 0x7fc1c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.503+0000 7fc1e6344700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc1c4066f30 0x7fc1c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.503+0000 7fc1d67fc700 1 -- 172.21.15.123:0/1133464767 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc1d8084940 con 0x7fc1e00c2c20 2023-10-06T04:31:45.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.503+0000 7fc1e6344700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc1c4066f30 0x7fc1c40693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc003cd0 tx=0x7fc1dc007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.507+0000 7fc1d67fc700 1 -- 172.21.15.123:0/1133464767 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc1d8051b10 con 0x7fc1e00c2c20 2023-10-06T04:31:45.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.731+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7fc1e006dae0 con 0x7fc1e00c2c20 2023-10-06T04:31:45.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.731+0000 7fc1d67fc700 1 -- 172.21.15.123:0/1133464767 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7fc1d801a040 con 0x7fc1e00c2c20 2023-10-06T04:31:45.734 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:45.734 INFO:teuthology.orchestra.run.smithi123.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-10-06T04:31:45.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc1c4066f30 msgr2=0x7fc1c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc1c4066f30 0x7fc1c40693e0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7fc1dc003cd0 tx=0x7fc1dc007d30 comp rx=0 tx=0).stop 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e00c2c20 msgr2=0x7fc1e0159de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e00c2c20 0x7fc1e0159de0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7fc1e00ca590 tx=0x7fc1d800b040 comp rx=0 tx=0).stop 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 shutdown_connections 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc1c4066f30 0x7fc1c40693e0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc1e00c2c20 0x7fc1e0159de0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc1cc004810 0x7fc1e0168cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 --2- 172.21.15.123:0/1133464767 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc1e0154b20 0x7fc1e00c9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 >> 172.21.15.123:0/1133464767 conn(0x7fc1e00ce0c0 msgr2=0x7fc1e00af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:45.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 shutdown_connections 2023-10-06T04:31:45.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.733+0000 7fc1e85a8700 1 -- 172.21.15.123:0/1133464767 wait complete. 2023-10-06T04:31:45.747 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-10-06T04:31:45.748 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:45.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.863+0000 7fdcc4017700 1 Processor -- start 2023-10-06T04:31:45.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.863+0000 7fdcc4017700 1 -- start start 2023-10-06T04:31:45.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc4017700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcbc068c90 0x7fdcbc156de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc4017700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc0cc190 con 0x7fdcbc157320 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc4017700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcbc06ab00 con 0x7fdcbc0688c0 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc4017700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcbc0cc620 con 0x7fdcbc068c90 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc15b2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdcbc068c90 0x7fdcbc156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc25b4700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdcbc157320 0x7fdcbc0c3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57040/0 (socket says 172.21.15.123:57040) 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc25b4700 1 -- 172.21.15.123:0/3852213162 learned_addr learned my addr 172.21.15.123:0/3852213162 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc15b2700 1 -- 172.21.15.123:0/3852213162 >> v1:172.21.15.177:6789/0 conn(0x7fdcbc0688c0 legacy=0x7fdcbc06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc15b2700 1 -- 172.21.15.123:0/3852213162 >> v1:172.21.15.123:6789/0 conn(0x7fdcbc157320 legacy=0x7fdcbc0c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc15b2700 1 -- 172.21.15.123:0/3852213162 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcbc0cb580 con 0x7fdcbc068c90 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc15b2700 1 --2- 172.21.15.123:0/3852213162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc068c90 0x7fdcbc156de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0098e0 tx=0x7fdcac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ea237ad5d644eea server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.864+0000 7fdcc0db1700 1 -- 172.21.15.123:0/3852213162 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac009e80 con 0x7fdcbc068c90 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc0db1700 1 -- 172.21.15.123:0/3852213162 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcac00b860 con 0x7fdcbc068c90 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc0db1700 1 -- 172.21.15.123:0/3852213162 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcac00bbe0 con 0x7fdcbc068c90 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc4017700 1 -- 172.21.15.123:0/3852213162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc068c90 msgr2=0x7fdcbc156de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc4017700 1 --2- 172.21.15.123:0/3852213162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc068c90 0x7fdcbc156de0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fdcac0098e0 tx=0x7fdcac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc4017700 1 -- 172.21.15.123:0/3852213162 shutdown_connections 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc4017700 1 --2- 172.21.15.123:0/3852213162 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc068c90 0x7fdcbc156de0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc4017700 1 -- 172.21.15.123:0/3852213162 >> 172.21.15.123:0/3852213162 conn(0x7fdcbc0c9c90 msgr2=0x7fdcbc06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:45.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc4017700 1 -- 172.21.15.123:0/3852213162 shutdown_connections 2023-10-06T04:31:45.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.865+0000 7fdcc4017700 1 -- 172.21.15.123:0/3852213162 wait complete. 2023-10-06T04:31:45.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.866+0000 7fdcc4017700 1 Processor -- start 2023-10-06T04:31:45.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.866+0000 7fdcc4017700 1 -- start start 2023-10-06T04:31:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc4017700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcbc0688c0 0x7fdcbc0ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc4017700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcbc068c90 0x7fdcbc0c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc4017700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc157320 0x7fdcbc14a290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc4017700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc0caa20 con 0x7fdcbc157320 2023-10-06T04:31:45.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc4017700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc16a140 con 0x7fdcbc068c90 2023-10-06T04:31:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc4017700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcbc169890 con 0x7fdcbc0688c0 2023-10-06T04:31:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc157320 0x7fdcbc14a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc157320 0x7fdcbc14a290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43516/0 (socket says 172.21.15.123:43516) 2023-10-06T04:31:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 -- 172.21.15.123:0/269077956 learned_addr learned my addr 172.21.15.123:0/269077956 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc15b2700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcbc068c90 0x7fdcbc0c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc1db3700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcbc0688c0 0x7fdcbc0ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 -- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcbc0688c0 msgr2=0x7fdcbc0ca060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcbc0688c0 0x7fdcbc0ca060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 -- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcbc068c90 msgr2=0x7fdcbc0c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcbc068c90 0x7fdcbc0c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 -- 172.21.15.123:0/269077956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcac009420 con 0x7fdcbc157320 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.867+0000 7fdcc25b4700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc157320 0x7fdcbc14a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400cbc0 tx=0x7fdcb40019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.868+0000 7fdcb1ffb700 1 -- 172.21.15.123:0/269077956 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb400a890 con 0x7fdcbc157320 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.868+0000 7fdcb1ffb700 1 -- 172.21.15.123:0/269077956 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb400a9f0 con 0x7fdcbc157320 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.868+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcbc14a7d0 con 0x7fdcbc157320 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.868+0000 7fdcb1ffb700 1 -- 172.21.15.123:0/269077956 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb400ad70 con 0x7fdcbc157320 2023-10-06T04:31:45.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.868+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcbc169af0 con 0x7fdcbc157320 2023-10-06T04:31:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.869+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcbc16a2b0 con 0x7fdcbc157320 2023-10-06T04:31:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.869+0000 7fdcb1ffb700 1 -- 172.21.15.123:0/269077956 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcb4007300 con 0x7fdcbc157320 2023-10-06T04:31:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.870+0000 7fdcb1ffb700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdca0066f30 0x7fdca00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.870+0000 7fdcc1db3700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdca0066f30 0x7fdca00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.870+0000 7fdcb1ffb700 1 -- 172.21.15.123:0/269077956 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdcb4084980 con 0x7fdcbc157320 2023-10-06T04:31:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.870+0000 7fdcc1db3700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdca0066f30 0x7fdca00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0b26a0 tx=0x7fdcb8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:45.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:45.874+0000 7fdcb1ffb700 1 -- 172.21.15.123:0/269077956 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcb4051b10 con 0x7fdcbc157320 2023-10-06T04:31:46.060 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.057+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdcbc04bb80 con 0x7fdcbc157320 2023-10-06T04:31:46.060 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.058+0000 7fdcb1ffb700 1 -- 172.21.15.123:0/269077956 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7fdcb4055130 con 0x7fdcbc157320 2023-10-06T04:31:46.061 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:46.061 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdca0066f30 msgr2=0x7fdca00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdca0066f30 0x7fdca00693e0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7fdcbc0b26a0 tx=0x7fdcb8009040 comp rx=0 tx=0).stop 2023-10-06T04:31:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc157320 msgr2=0x7fdcbc14a290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc157320 0x7fdcbc14a290 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fdcb400cbc0 tx=0x7fdcb40019c0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 shutdown_connections 2023-10-06T04:31:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdca0066f30 0x7fdca00693e0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcbc157320 0x7fdcbc14a290 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdcbc0688c0 0x7fdcbc0ca060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 --2- 172.21.15.123:0/269077956 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcbc068c90 0x7fdcbc0c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 >> 172.21.15.123:0/269077956 conn(0x7fdcbc0c9c90 msgr2=0x7fdcbc0c2bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 shutdown_connections 2023-10-06T04:31:46.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.060+0000 7fdcc4017700 1 -- 172.21.15.123:0/269077956 wait complete. 2023-10-06T04:31:46.074 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:31:32.045143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:31:33.390421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:31:46.075 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:31:46.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 Processor -- start 2023-10-06T04:31:46.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 -- start start 2023-10-06T04:31:46.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d740c8340 0x7f5d740a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5d740c8710 0x7f5d740c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d74068670 0x7f5d74068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d74069050 con 0x7f5d74068670 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d74069aa0 con 0x7f5d740c8710 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.192+0000 7f5d79cab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d7406a340 con 0x7f5d740c8340 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d74068670 0x7f5d74068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d74068670 0x7f5d74068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43520/0 (socket says 172.21.15.123:43520) 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 -- 172.21.15.123:0/2220326817 learned_addr learned my addr 172.21.15.123:0/2220326817 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d72ffd700 1 --2- 172.21.15.123:0/2220326817 >> v2:172.21.15.177:3300/0 conn(0x7f5d740c8710 0x7f5d740c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d737fe700 1 --2- 172.21.15.123:0/2220326817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d740c8340 0x7f5d740a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 -- 172.21.15.123:0/2220326817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d740c8340 msgr2=0x7f5d740a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:46.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 --2- 172.21.15.123:0/2220326817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d740c8340 0x7f5d740a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 -- 172.21.15.123:0/2220326817 >> v2:172.21.15.177:3300/0 conn(0x7f5d740c8710 msgr2=0x7f5d740c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 --2- 172.21.15.123:0/2220326817 >> v2:172.21.15.177:3300/0 conn(0x7f5d740c8710 0x7f5d740c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 -- 172.21.15.123:0/2220326817 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d740aa6f0 con 0x7f5d74068670 2023-10-06T04:31:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d72ffd700 1 --2- 172.21.15.123:0/2220326817 >> v2:172.21.15.177:3300/0 conn(0x7f5d740c8710 0x7f5d740c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:46.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d737fe700 1 --2- 172.21.15.123:0/2220326817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d740c8340 0x7f5d740a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d73fff700 1 --2- 172.21.15.123:0/2220326817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d74068670 0x7f5d74068b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009f20 tx=0x7f5d6c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462ccbe6354af110 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d727fc700 1 -- 172.21.15.123:0/2220326817 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c00e070 con 0x7f5d74068670 2023-10-06T04:31:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d727fc700 1 -- 172.21.15.123:0/2220326817 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c00ba40 con 0x7f5d74068670 2023-10-06T04:31:46.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.193+0000 7f5d727fc700 1 -- 172.21.15.123:0/2220326817 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c0092e0 con 0x7f5d74068670 2023-10-06T04:31:46.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 -- 172.21.15.123:0/2220326817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d74068670 msgr2=0x7f5d74068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 --2- 172.21.15.123:0/2220326817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d74068670 0x7f5d74068b10 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009f20 tx=0x7f5d6c009840 comp rx=0 tx=0).stop 2023-10-06T04:31:46.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 -- 172.21.15.123:0/2220326817 shutdown_connections 2023-10-06T04:31:46.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 --2- 172.21.15.123:0/2220326817 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d74068670 0x7f5d74068b10 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 --2- 172.21.15.123:0/2220326817 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d740c8340 0x7f5d740a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 --2- 172.21.15.123:0/2220326817 >> v2:172.21.15.177:3300/0 conn(0x7f5d740c8710 0x7f5d740c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 -- 172.21.15.123:0/2220326817 >> 172.21.15.123:0/2220326817 conn(0x7f5d740c9ca0 msgr2=0x7f5d7406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:46.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 -- 172.21.15.123:0/2220326817 shutdown_connections 2023-10-06T04:31:46.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.194+0000 7f5d79cab700 1 -- 172.21.15.123:0/2220326817 wait complete. 2023-10-06T04:31:46.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.195+0000 7f5d79cab700 1 Processor -- start 2023-10-06T04:31:46.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.195+0000 7f5d79cab700 1 -- start start 2023-10-06T04:31:46.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d74068670 0x7f5d740c76f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d740c8340 0x7f5d74157f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d740c8710 0x7f5d740c47f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740c6420 con 0x7f5d740c8710 2023-10-06T04:31:46.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740c7260 con 0x7f5d740c8340 2023-10-06T04:31:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d740c6650 con 0x7f5d74068670 2023-10-06T04:31:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d740c8710 0x7f5d740c47f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d740c8710 0x7f5d740c47f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43532/0 (socket says 172.21.15.123:43532) 2023-10-06T04:31:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 -- 172.21.15.123:0/771125026 learned_addr learned my addr 172.21.15.123:0/771125026 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d72ffd700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d740c8340 0x7f5d74157f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d737fe700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d74068670 0x7f5d740c76f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 -- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d74068670 msgr2=0x7f5d740c76f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d74068670 0x7f5d740c76f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 -- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d740c8340 msgr2=0x7f5d74157f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d740c8340 0x7f5d74157f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 -- 172.21.15.123:0/771125026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d6c009ae0 con 0x7f5d740c8710 2023-10-06T04:31:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d72ffd700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d740c8340 0x7f5d74157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d737fe700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d74068670 0x7f5d740c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d73fff700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d740c8710 0x7f5d740c47f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009880 tx=0x7f5d6c018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d6b7fe700 1 -- 172.21.15.123:0/771125026 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c008e40 con 0x7f5d740c8710 2023-10-06T04:31:46.209 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d6b7fe700 1 -- 172.21.15.123:0/771125026 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d6c026460 con 0x7f5d740c8710 2023-10-06T04:31:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d740c6930 con 0x7f5d740c8710 2023-10-06T04:31:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d6b7fe700 1 -- 172.21.15.123:0/771125026 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d6c026820 con 0x7f5d740c8710 2023-10-06T04:31:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.196+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d740c6f70 con 0x7f5d740c8710 2023-10-06T04:31:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.197+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d741584d0 con 0x7f5d740c8710 2023-10-06T04:31:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.198+0000 7f5d6b7fe700 1 -- 172.21.15.123:0/771125026 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d6c003040 con 0x7f5d740c8710 2023-10-06T04:31:46.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.198+0000 7f5d6b7fe700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d54066f30 0x7f5d540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.198+0000 7f5d737fe700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d54066f30 0x7f5d540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.198+0000 7f5d6b7fe700 1 -- 172.21.15.123:0/771125026 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5d6c086890 con 0x7f5d740c8710 2023-10-06T04:31:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.199+0000 7f5d737fe700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d54066f30 0x7f5d540693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d64008a80 tx=0x7f5d64008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.203+0000 7f5d6b7fe700 1 -- 172.21.15.123:0/771125026 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d6c053a50 con 0x7f5d740c8710 2023-10-06T04:31:46.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.384+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5d740ca0a0 con 0x7f5d740c8710 2023-10-06T04:31:46.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.384+0000 7f5d6b7fe700 1 -- 172.21.15.123:0/771125026 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5d6c057070 con 0x7f5d740c8710 2023-10-06T04:31:46.387 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:31:46.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d54066f30 msgr2=0x7f5d540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d54066f30 0x7f5d540693e0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f5d64008a80 tx=0x7f5d64008040 comp rx=0 tx=0).stop 2023-10-06T04:31:46.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d740c8710 msgr2=0x7f5d740c47f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d740c8710 0x7f5d740c47f0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f5d6c009880 tx=0x7f5d6c018000 comp rx=0 tx=0).stop 2023-10-06T04:31:46.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 shutdown_connections 2023-10-06T04:31:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d54066f30 0x7f5d540693e0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d740c8710 0x7f5d740c47f0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d74068670 0x7f5d740c76f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 --2- 172.21.15.123:0/771125026 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d740c8340 0x7f5d74157f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.386+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 >> 172.21.15.123:0/771125026 conn(0x7f5d740c9ca0 msgr2=0x7f5d74158880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.387+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 shutdown_connections 2023-10-06T04:31:46.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.387+0000 7f5d79cab700 1 -- 172.21.15.123:0/771125026 wait complete. 2023-10-06T04:31:46.402 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:31:46.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.519+0000 7fdd0ab64700 1 Processor -- start 2023-10-06T04:31:46.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.519+0000 7fdd0ab64700 1 -- start start 2023-10-06T04:31:46.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd0ab64700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdd040c8710 0x7fdd040c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd0ab64700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdd04068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd0ab64700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd04069aa0 con 0x7fdd040c8340 2023-10-06T04:31:46.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd0ab64700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd040aa500 con 0x7fdd040c8710 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd0ab64700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04069050 con 0x7fdd04068670 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd08900700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fdd040c8340 0x7fdd040a9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57042/0 (socket says 172.21.15.123:57042) 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd08900700 1 -- 172.21.15.123:0/973833699 learned_addr learned my addr 172.21.15.123:0/973833699 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd09101700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdd04068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd03fff700 1 --2- 172.21.15.123:0/973833699 >> v2:172.21.15.177:3300/0 conn(0x7fdd040c8710 0x7fdd040c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 568747857 0 0) 0x7fdd04069aa0 con 0x7fdd040c8340 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.520+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcf4003570 con 0x7fdd040c8340 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3105431581 0 0) 0x7fdcf4003570 con 0x7fdd040c8340 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 msgr2=0x7fdd04068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdd04068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 >> v2:172.21.15.177:3300/0 conn(0x7fdd040c8710 msgr2=0x7fdd040c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:46.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 --2- 172.21.15.123:0/973833699 >> v2:172.21.15.177:3300/0 conn(0x7fdd040c8710 0x7fdd040c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd0406a340 con 0x7fdd040c8340 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fdcf8001470 con 0x7fdd040c8340 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd09101700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdd04068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 >> v1:172.21.15.123:6789/0 conn(0x7fdd040c8340 legacy=0x7fdd040a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf4003a60 0x7fdcf4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4004810 0x7fdcf4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdcf40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04069050 con 0x7fdcf4003a60 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf400ba60 con 0x7fdcf4004810 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcf400bba0 con 0x7fdd04068670 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd03fff700 1 --2- 172.21.15.123:0/973833699 >> v2:172.21.15.177:3300/0 conn(0x7fdd040c8710 0x7fdd040c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:31:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd08900700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf4003a60 0x7fdcf4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd09101700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdcf40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd03fff700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4004810 0x7fdcf4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd08900700 1 -- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 msgr2=0x7fdcf40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd08900700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdcf40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd08900700 1 -- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4004810 msgr2=0x7fdcf4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd08900700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4004810 0x7fdcf4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd08900700 1 -- 172.21.15.123:0/973833699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcfc009a20 con 0x7fdcf4003a60 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd09101700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd04068670 0x7fdcf40051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd03fff700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4004810 0x7fdcf4004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd08900700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf4003a60 0x7fdcf4003e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c3c0 tx=0x7fdd0406a2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f379b129a6f052c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf800cc90 con 0x7fdcf4003a60 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf800ce30 con 0x7fdcf4003a60 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.521+0000 7fdd037fe700 1 -- 172.21.15.123:0/973833699 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf8005b70 con 0x7fdcf4003a60 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.522+0000 7fdd0ab64700 1 -- 172.21.15.123:0/973833699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf4003a60 msgr2=0x7fdcf4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.522+0000 7fdd0ab64700 1 --2- 172.21.15.123:0/973833699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdcf4003a60 0x7fdcf4003e70 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c3c0 tx=0x7fdd0406a2b0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.522+0000 7fdd09101700 1 -- 172.21.15.123:0/973833699 reap_dead start 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.522+0000 7fdd0ab64700 1 -- 172.21.15.123:0/973833699 shutdown_connections 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.522+0000 7fdd0ab64700 1 -- 172.21.15.123:0/973833699 >> 172.21.15.123:0/973833699 conn(0x7fdd040c9ca0 msgr2=0x7fdd0406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.522+0000 7fdd0ab64700 1 -- 172.21.15.123:0/973833699 shutdown_connections 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.522+0000 7fdd0ab64700 1 -- 172.21.15.123:0/973833699 wait complete. 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.523+0000 7fdd0ab64700 1 Processor -- start 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.523+0000 7fdd0ab64700 1 -- start start 2023-10-06T04:31:46.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd0ab64700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4003a60 0x7fdd04156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd0ab64700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd0415e560 0x7fdd0415e9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd0ab64700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd040caeb0 0x7fdd040cb320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd0ab64700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd0415ef10 con 0x7fdd0415e560 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd0ab64700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd040cb860 con 0x7fdcf4003a60 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd0ab64700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd04144740 con 0x7fdd040caeb0 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd0415e560 0x7fdd0415e9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd0415e560 0x7fdd0415e9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43544/0 (socket says 172.21.15.123:43544) 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 -- 172.21.15.123:0/1257603939 learned_addr learned my addr 172.21.15.123:0/1257603939 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd08900700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4003a60 0x7fdd04156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd09101700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd040caeb0 0x7fdd040cb320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 -- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd040caeb0 msgr2=0x7fdd040cb320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd040caeb0 0x7fdd040cb320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 -- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4003a60 msgr2=0x7fdd04156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4003a60 0x7fdd04156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 -- 172.21.15.123:0/1257603939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd040ac2e0 con 0x7fdd0415e560 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd03fff700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd0415e560 0x7fdd0415e9d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0011ab0 tx=0x7fdcf0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.524+0000 7fdd00ff9700 1 -- 172.21.15.123:0/1257603939 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000f860 con 0x7fdd0415e560 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.525+0000 7fdd00ff9700 1 -- 172.21.15.123:0/1257603939 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcf000f9c0 con 0x7fdd0415e560 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.525+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd040ace60 con 0x7fdd0415e560 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.525+0000 7fdd00ff9700 1 -- 172.21.15.123:0/1257603939 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcf000fd40 con 0x7fdd0415e560 2023-10-06T04:31:46.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.525+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd040ac530 con 0x7fdd0415e560 2023-10-06T04:31:46.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.525+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd040acff0 con 0x7fdd0415e560 2023-10-06T04:31:46.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.526+0000 7fdd00ff9700 1 -- 172.21.15.123:0/1257603939 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdcf0015670 con 0x7fdd0415e560 2023-10-06T04:31:46.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.526+0000 7fdd00ff9700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdce8066f30 0x7fdce80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.526+0000 7fdd00ff9700 1 -- 172.21.15.123:0/1257603939 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fdcf0084ea0 con 0x7fdd0415e560 2023-10-06T04:31:46.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.526+0000 7fdd08900700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdce8066f30 0x7fdce80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.527+0000 7fdd08900700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdce8066f30 0x7fdce80693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c3c0 tx=0x7fdcf8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.530+0000 7fdd00ff9700 1 -- 172.21.15.123:0/1257603939 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdcf0051fb0 con 0x7fdd0415e560 2023-10-06T04:31:46.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.711+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fdd040aa6f0 con 0x7fdd0415e560 2023-10-06T04:31:46.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.712+0000 7fdd00ff9700 1 -- 172.21.15.123:0/1257603939 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fdcf000fea0 con 0x7fdd0415e560 2023-10-06T04:31:46.714 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:31:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdce8066f30 msgr2=0x7fdce80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdce8066f30 0x7fdce80693e0 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fdcf800c3c0 tx=0x7fdcf8015000 comp rx=0 tx=0).stop 2023-10-06T04:31:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd0415e560 msgr2=0x7fdd0415e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd0415e560 0x7fdd0415e9d0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fdcf0011ab0 tx=0x7fdcf0008040 comp rx=0 tx=0).stop 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 shutdown_connections 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdce8066f30 0x7fdce80693e0 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdd040caeb0 0x7fdd040cb320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdcf4003a60 0x7fdd04156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 --2- 172.21.15.123:0/1257603939 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdd0415e560 0x7fdd0415e9d0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 >> 172.21.15.123:0/1257603939 conn(0x7fdd040c9ca0 msgr2=0x7fdd040cc020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 shutdown_connections 2023-10-06T04:31:46.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.714+0000 7fdd0ab64700 1 -- 172.21.15.123:0/1257603939 wait complete. 2023-10-06T04:31:46.728 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:31:46.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.845+0000 7f038f1e5700 1 Processor -- start 2023-10-06T04:31:46.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.845+0000 7f038f1e5700 1 -- start start 2023-10-06T04:31:46.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.845+0000 7f038f1e5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0388068d80 0x7f03880c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.845+0000 7f038f1e5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0388069aa0 con 0x7f03880c8340 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.845+0000 7f038f1e5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f038806a340 con 0x7f03880c8710 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.845+0000 7f038f1e5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03880aa500 con 0x7f0388068d80 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038d782700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0388068d80 0x7f03880c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038d782700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0388068d80 0x7f03880c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43558/0 (socket says 172.21.15.123:43558) 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038cf81700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f03880c8340 0x7f03880a9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57056/0 (socket says 172.21.15.123:57056) 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038d782700 1 -- 172.21.15.123:0/1812730193 learned_addr learned my addr 172.21.15.123:0/1812730193 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038d782700 1 -- 172.21.15.123:0/1812730193 >> v1:172.21.15.177:6789/0 conn(0x7f03880c8710 legacy=0x7f0388068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038d782700 1 -- 172.21.15.123:0/1812730193 >> v1:172.21.15.123:6789/0 conn(0x7f03880c8340 legacy=0x7f03880a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038d782700 1 -- 172.21.15.123:0/1812730193 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03880aa6f0 con 0x7f0388068d80 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f03877fe700 1 -- 172.21.15.123:0/1812730193 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 459261823 0 0) 0x7f0388069aa0 con 0x7f03880c8340 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f038d782700 1 --2- 172.21.15.123:0/1812730193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c8b60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0380009f20 tx=0x7f0380009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0a14f059a136a75 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f03877fe700 1 -- 172.21.15.123:0/1812730193 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f038000e070 con 0x7f0388068d80 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f03877fe700 1 -- 172.21.15.123:0/1812730193 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f038000ba40 con 0x7f0388068d80 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.846+0000 7f03877fe700 1 -- 172.21.15.123:0/1812730193 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03800092e0 con 0x7f0388068d80 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.847+0000 7f038f1e5700 1 -- 172.21.15.123:0/1812730193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 msgr2=0x7f03880c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.847+0000 7f038f1e5700 1 --2- 172.21.15.123:0/1812730193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c8b60 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0380009f20 tx=0x7f0380009840 comp rx=0 tx=0).stop 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.847+0000 7f038f1e5700 1 -- 172.21.15.123:0/1812730193 shutdown_connections 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.847+0000 7f038f1e5700 1 --2- 172.21.15.123:0/1812730193 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c8b60 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.847+0000 7f038f1e5700 1 -- 172.21.15.123:0/1812730193 >> 172.21.15.123:0/1812730193 conn(0x7f03880c9ca0 msgr2=0x7f038806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:46.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.847+0000 7f038f1e5700 1 -- 172.21.15.123:0/1812730193 shutdown_connections 2023-10-06T04:31:46.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.847+0000 7f038f1e5700 1 -- 172.21.15.123:0/1812730193 wait complete. 2023-10-06T04:31:46.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.848+0000 7f038f1e5700 1 Processor -- start 2023-10-06T04:31:46.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.848+0000 7f038f1e5700 1 -- start start 2023-10-06T04:31:46.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c13f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03880c8340 0x7f038815a740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03880c8710 0x7f038815ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03881575b0 con 0x7f0388068d80 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03880c0100 con 0x7f03880c8710 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03880c0f60 con 0x7f03880c8340 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c13f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c13f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43572/0 (socket says 172.21.15.123:43572) 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 -- 172.21.15.123:0/3080355151 learned_addr learned my addr 172.21.15.123:0/3080355151 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 -- 172.21.15.123:0/3080355151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03880c8340 msgr2=0x7f038815a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03880c8340 0x7f038815a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 -- 172.21.15.123:0/3080355151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03880c8710 msgr2=0x7f038815ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038d782700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03880c8710 0x7f038815ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03880c8710 0x7f038815ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:46.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 -- 172.21.15.123:0/3080355151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0380009ae0 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038cf81700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c13f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f037c0011e0 tx=0x7f037c007e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f0384ff9700 1 -- 172.21.15.123:0/3080355151 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c007300 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f0384ff9700 1 -- 172.21.15.123:0/3080355151 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f037c007460 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03880af170 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f0384ff9700 1 -- 172.21.15.123:0/3080355151 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f037c004830 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.849+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03880c0630 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.850+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0388069cf0 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.851+0000 7f0384ff9700 1 -- 172.21.15.123:0/3080355151 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f037c004be0 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.851+0000 7f0384ff9700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f036c066f30 0x7f036c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.851+0000 7f0384ff9700 1 -- 172.21.15.123:0/3080355151 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f037c083bd0 con 0x7f0388068d80 2023-10-06T04:31:46.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.851+0000 7f0387fff700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f036c066f30 0x7f036c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:46.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.851+0000 7f0387fff700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f036c066f30 0x7f036c0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f038806b890 tx=0x7f0374006c60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:46.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:46.855+0000 7f0384ff9700 1 -- 172.21.15.123:0/3080355151 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f037c050d60 con 0x7f0388068d80 2023-10-06T04:31:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.034+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03880cc510 con 0x7f0388068d80 2023-10-06T04:31:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.035+0000 7f0384ff9700 1 -- 172.21.15.123:0/3080355151 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f037c0180a0 con 0x7f0388068d80 2023-10-06T04:31:47.038 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:31:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.036+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f036c066f30 msgr2=0x7f036c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.036+0000 7f038f1e5700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f036c066f30 0x7f036c0693e0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f038806b890 tx=0x7f0374006c60 comp rx=0 tx=0).stop 2023-10-06T04:31:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.036+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 msgr2=0x7f03880c13f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.036+0000 7f038f1e5700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c13f0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f037c0011e0 tx=0x7f037c007e10 comp rx=0 tx=0).stop 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 shutdown_connections 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f036c066f30 0x7f036c0693e0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0388068d80 0x7f03880c13f0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03880c8710 0x7f038815ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 --2- 172.21.15.123:0/3080355151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03880c8340 0x7f038815a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 >> 172.21.15.123:0/3080355151 conn(0x7f03880c9ca0 msgr2=0x7f03880cac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 shutdown_connections 2023-10-06T04:31:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.037+0000 7f038f1e5700 1 -- 172.21.15.123:0/3080355151 wait complete. 2023-10-06T04:31:47.051 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:31:47.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.168+0000 7f794bd7f700 1 Processor -- start 2023-10-06T04:31:47.171 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794bd7f700 1 -- start start 2023-10-06T04:31:47.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794bd7f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7944157080 0x7f79440abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794bd7f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79440ac470 0x7f7944068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794bd7f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7944157450 0x7f7944068bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794bd7f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79440c7810 con 0x7f7944157450 2023-10-06T04:31:47.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794bd7f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79440ac9f0 con 0x7f7944157080 2023-10-06T04:31:47.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794bd7f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79440c6ee0 con 0x7f79440ac470 2023-10-06T04:31:47.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794a31c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7944157450 0x7f7944068bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794a31c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f7944157450 0x7f7944068bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43576/0 (socket says 172.21.15.123:43576) 2023-10-06T04:31:47.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.169+0000 7f794a31c700 1 -- 172.21.15.123:0/263989854 learned_addr learned my addr 172.21.15.123:0/263989854 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:47.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f7949b1b700 1 --2- 172.21.15.123:0/263989854 >> v2:172.21.15.177:3300/0 conn(0x7f7944157080 0x7f79440abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794931a700 1 --2- 172.21.15.123:0/263989854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79440ac470 0x7f7944068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794a31c700 1 -- 172.21.15.123:0/263989854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79440ac470 msgr2=0x7f7944068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:47.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794a31c700 1 --2- 172.21.15.123:0/263989854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79440ac470 0x7f7944068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794a31c700 1 -- 172.21.15.123:0/263989854 >> v2:172.21.15.177:3300/0 conn(0x7f7944157080 msgr2=0x7f79440abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794a31c700 1 --2- 172.21.15.123:0/263989854 >> v2:172.21.15.177:3300/0 conn(0x7f7944157080 0x7f79440abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794a31c700 1 -- 172.21.15.123:0/263989854 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79440c71c0 con 0x7f7944157450 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f7949b1b700 1 --2- 172.21.15.123:0/263989854 >> v2:172.21.15.177:3300/0 conn(0x7f7944157080 0x7f79440abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794a31c700 1 --2- 172.21.15.123:0/263989854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7944157450 0x7f7944068bb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f794406d1c0 tx=0x7f793c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ecf873e5a43828 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f7948b19700 1 -- 172.21.15.123:0/263989854 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c00e070 con 0x7f7944157450 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f7948b19700 1 -- 172.21.15.123:0/263989854 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f793c00ba50 con 0x7f7944157450 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f7948b19700 1 -- 172.21.15.123:0/263989854 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f793c0092e0 con 0x7f7944157450 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794bd7f700 1 -- 172.21.15.123:0/263989854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7944157450 msgr2=0x7f7944068bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.170+0000 7f794bd7f700 1 --2- 172.21.15.123:0/263989854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7944157450 0x7f7944068bb0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f794406d1c0 tx=0x7f793c009840 comp rx=0 tx=0).stop 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.171+0000 7f794bd7f700 1 -- 172.21.15.123:0/263989854 shutdown_connections 2023-10-06T04:31:47.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.171+0000 7f794bd7f700 1 --2- 172.21.15.123:0/263989854 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7944157450 0x7f7944068bb0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.171+0000 7f794bd7f700 1 --2- 172.21.15.123:0/263989854 >> v2:172.21.15.177:3300/0 conn(0x7f7944157080 0x7f79440abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.171+0000 7f794bd7f700 1 --2- 172.21.15.123:0/263989854 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f79440ac470 0x7f7944068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.171+0000 7f794bd7f700 1 -- 172.21.15.123:0/263989854 >> 172.21.15.123:0/263989854 conn(0x7f79440c9ca0 msgr2=0x7f794406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:47.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.171+0000 7f794bd7f700 1 -- 172.21.15.123:0/263989854 shutdown_connections 2023-10-06T04:31:47.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.171+0000 7f794bd7f700 1 -- 172.21.15.123:0/263989854 wait complete. 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 Processor -- start 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 -- start start 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79440ac470 0x7f79441642e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7944157080 0x7f7944157960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7944157450 0x7f7944157ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794415a690 con 0x7f79440ac470 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f794415a970 con 0x7f7944157080 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.172+0000 7f794bd7f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7944154590 con 0x7f7944157450 2023-10-06T04:31:47.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79440ac470 0x7f79441642e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79440ac470 0x7f79441642e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43586/0 (socket says 172.21.15.123:43586) 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 -- 172.21.15.123:0/283116864 learned_addr learned my addr 172.21.15.123:0/283116864 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f794931a700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7944157080 0x7f7944157960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f794a31c700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7944157450 0x7f7944157ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 -- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7944157450 msgr2=0x7f7944157ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7944157450 0x7f7944157ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 -- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7944157080 msgr2=0x7f7944157960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7944157080 0x7f7944157960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 -- 172.21.15.123:0/283116864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f793c009ae0 con 0x7f79440ac470 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7949b1b700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79440ac470 0x7f79441642e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f793400e8d0 tx=0x7f7934008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7939ffb700 1 -- 172.21.15.123:0/283116864 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934013410 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7939ffb700 1 -- 172.21.15.123:0/283116864 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7934013570 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f7939ffb700 1 -- 172.21.15.123:0/283116864 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7934013910 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7944154870 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.173+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79441649e0 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.174+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f794415ab90 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.174+0000 7f7939ffb700 1 -- 172.21.15.123:0/283116864 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7934013d40 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.175+0000 7f7939ffb700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.175+0000 7f7939ffb700 1 -- 172.21.15.123:0/283116864 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7934084200 con 0x7f79440ac470 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.175+0000 7f794931a700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.175+0000 7f794931a700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7928066f30 0x7f79280693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7940006fd0 tx=0x7f7940008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.179+0000 7f7939ffb700 1 -- 172.21.15.123:0/283116864 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7934051310 con 0x7f79440ac470 2023-10-06T04:31:47.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.362+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79441649e0 con 0x7f79440ac470 2023-10-06T04:31:47.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.362+0000 7f7939ffb700 1 -- 172.21.15.123:0/283116864 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f79441649e0 con 0x7f79440ac470 2023-10-06T04:31:47.365 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:31:47.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.364+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7928066f30 msgr2=0x7f79280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.364+0000 7f794bd7f700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7928066f30 0x7f79280693e0 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f7940006fd0 tx=0x7f7940008040 comp rx=0 tx=0).stop 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.364+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79440ac470 msgr2=0x7f79441642e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.364+0000 7f794bd7f700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79440ac470 0x7f79441642e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f793400e8d0 tx=0x7f7934008040 comp rx=0 tx=0).stop 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 shutdown_connections 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7928066f30 0x7f79280693e0 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f79440ac470 0x7f79441642e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7944157450 0x7f7944157ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 --2- 172.21.15.123:0/283116864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7944157080 0x7f7944157960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 >> 172.21.15.123:0/283116864 conn(0x7f79440c9ca0 msgr2=0x7f79440cac30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:47.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 shutdown_connections 2023-10-06T04:31:47.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.365+0000 7f794bd7f700 1 -- 172.21.15.123:0/283116864 wait complete. 2023-10-06T04:31:47.379 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:31:47.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.493+0000 7f950f9b4700 1 Processor -- start 2023-10-06T04:31:47.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.493+0000 7f950f9b4700 1 -- start start 2023-10-06T04:31:47.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.494+0000 7f950f9b4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9508156f60 0x7f950815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.494+0000 7f950f9b4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9508157910 con 0x7f95080ade40 2023-10-06T04:31:47.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.494+0000 7f950f9b4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f950806d240 con 0x7f950815b290 2023-10-06T04:31:47.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.494+0000 7f950f9b4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f950806d520 con 0x7f9508156f60 2023-10-06T04:31:47.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.494+0000 7f950d750700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9508156f60 0x7f950815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.494+0000 7f950df51700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f95080ade40 0x7f9508157da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57062/0 (socket says 172.21.15.123:57062) 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f950df51700 1 -- 172.21.15.123:0/3671253571 learned_addr learned my addr 172.21.15.123:0/3671253571 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f950d750700 1 -- 172.21.15.123:0/3671253571 >> v1:172.21.15.177:6789/0 conn(0x7f950815b290 legacy=0x7f95080ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f950d750700 1 -- 172.21.15.123:0/3671253571 >> v1:172.21.15.123:6789/0 conn(0x7f95080ade40 legacy=0x7f9508157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f94fffff700 1 -- 172.21.15.123:0/3671253571 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2974015295 0 0) 0x7f9508157910 con 0x7f95080ade40 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f950d750700 1 -- 172.21.15.123:0/3671253571 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f950806d800 con 0x7f9508156f60 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f950d750700 1 --2- 172.21.15.123:0/3671253571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f950815ad50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009910 tx=0x7f94f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffe7ea7aa60a9130 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f94fffff700 1 -- 172.21.15.123:0/3671253571 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800b7e0 con 0x7f9508156f60 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f94fffff700 1 -- 172.21.15.123:0/3671253571 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94f800b940 con 0x7f9508156f60 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f94fffff700 1 -- 172.21.15.123:0/3671253571 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94f800bce0 con 0x7f9508156f60 2023-10-06T04:31:47.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.495+0000 7f950f9b4700 1 -- 172.21.15.123:0/3671253571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 msgr2=0x7f950815ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.496+0000 7f950f9b4700 1 --2- 172.21.15.123:0/3671253571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f950815ad50 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f94f8009910 tx=0x7f94f80091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.496+0000 7f950f9b4700 1 -- 172.21.15.123:0/3671253571 shutdown_connections 2023-10-06T04:31:47.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.496+0000 7f950f9b4700 1 --2- 172.21.15.123:0/3671253571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f950815ad50 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.496+0000 7f950f9b4700 1 -- 172.21.15.123:0/3671253571 >> 172.21.15.123:0/3671253571 conn(0x7f95080ce0c0 msgr2=0x7f95080ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:47.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.496+0000 7f950f9b4700 1 -- 172.21.15.123:0/3671253571 shutdown_connections 2023-10-06T04:31:47.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.496+0000 7f950f9b4700 1 -- 172.21.15.123:0/3671253571 wait complete. 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.497+0000 7f950f9b4700 1 Processor -- start 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.497+0000 7f950f9b4700 1 -- start start 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95080ade40 0x7f95080b34c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f95080698d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f950815b290 0x7f95081544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c59a0 con 0x7f9508156f60 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c5c80 con 0x7f950815b290 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95080c5f60 con 0x7f95080ade40 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f95080698d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f95080698d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43616/0 (socket says 172.21.15.123:43616) 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 -- 172.21.15.123:0/2760453097 learned_addr learned my addr 172.21.15.123:0/2760453097 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950d750700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95080ade40 0x7f95080b34c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950df51700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f950815b290 0x7f95081544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 -- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95080ade40 msgr2=0x7f95080b34c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95080ade40 0x7f95080b34c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 -- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f950815b290 msgr2=0x7f95081544b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f950815b290 0x7f95081544b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 -- 172.21.15.123:0/2760453097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94f8009420 con 0x7f9508156f60 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950df51700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f950815b290 0x7f95081544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:47.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950cf4f700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f95080698d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f950400b5b0 tx=0x7f95040090f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f94fd7fa700 1 -- 172.21.15.123:0/2760453097 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950400f510 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f94fd7fa700 1 -- 172.21.15.123:0/2760453097 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950400f670 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95080c6240 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f94fd7fa700 1 -- 172.21.15.123:0/2760453097 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950400fa30 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.498+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95080c6800 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.499+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f950806e690 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.500+0000 7f94fd7fa700 1 -- 172.21.15.123:0/2760453097 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f950401a400 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.500+0000 7f94fd7fa700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f94ec066f30 0x7f94ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.500+0000 7f94fd7fa700 1 -- 172.21.15.123:0/2760453097 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9504084130 con 0x7f9508156f60 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.500+0000 7f950d750700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f94ec066f30 0x7f94ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.501+0000 7f950d750700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f94ec066f30 0x7f94ec0693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f94f80098e0 tx=0x7f94f8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.504+0000 7f94fd7fa700 1 -- 172.21.15.123:0/2760453097 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f95040512c0 con 0x7f9508156f60 2023-10-06T04:31:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.726+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f950806dae0 con 0x7f9508156f60 2023-10-06T04:31:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.726+0000 7f94fd7fa700 1 -- 172.21.15.123:0/2760453097 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f95040548e0 con 0x7f9508156f60 2023-10-06T04:31:47.729 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:47.729 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:31:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f94ec066f30 msgr2=0x7f94ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f94ec066f30 0x7f94ec0693e0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f94f80098e0 tx=0x7f94f8016000 comp rx=0 tx=0).stop 2023-10-06T04:31:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 msgr2=0x7f95080698d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f95080698d0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f950400b5b0 tx=0x7f95040090f0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 shutdown_connections 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f94ec066f30 0x7f94ec0693e0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f950815b290 0x7f95081544b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f95080ade40 0x7f95080b34c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 --2- 172.21.15.123:0/2760453097 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9508156f60 0x7f95080698d0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 >> 172.21.15.123:0/2760453097 conn(0x7f95080ce0c0 msgr2=0x7f9508155e10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 shutdown_connections 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.728+0000 7f950f9b4700 1 -- 172.21.15.123:0/2760453097 wait complete. 2023-10-06T04:31:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:31:47.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:47.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.856+0000 7fabd87f1700 1 Processor -- start 2023-10-06T04:31:47.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.857+0000 7fabd87f1700 1 -- start start 2023-10-06T04:31:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.857+0000 7fabd87f1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fabd0158f70 0x7fabd01593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.857+0000 7fabd87f1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd0069a30 con 0x7fabd0068c90 2023-10-06T04:31:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.857+0000 7fabd87f1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd006a890 con 0x7fabd00688c0 2023-10-06T04:31:47.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.857+0000 7fabd87f1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd0159930 con 0x7fabd0158f70 2023-10-06T04:31:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd6d8e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fabd0158f70 0x7fabd01593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd5d8c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fabd0068c90 0x7fabd006e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57076/0 (socket says 172.21.15.123:57076) 2023-10-06T04:31:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd5d8c700 1 -- 172.21.15.123:0/1080501856 learned_addr learned my addr 172.21.15.123:0/1080501856 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd6d8e700 1 -- 172.21.15.123:0/1080501856 >> v1:172.21.15.177:6789/0 conn(0x7fabd00688c0 legacy=0x7fabd00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd6d8e700 1 -- 172.21.15.123:0/1080501856 >> v1:172.21.15.123:6789/0 conn(0x7fabd0068c90 legacy=0x7fabd006e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.862 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd558b700 1 -- 172.21.15.123:0/1080501856 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 334815179 0 0) 0x7fabd0069a30 con 0x7fabd0068c90 2023-10-06T04:31:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd6d8e700 1 -- 172.21.15.123:0/1080501856 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0069c80 con 0x7fabd0158f70 2023-10-06T04:31:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd6d8e700 1 --2- 172.21.15.123:0/1080501856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0158f70 0x7fabd01593f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009e60 tx=0x7fabc8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f077584994b0ea8a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd558b700 1 -- 172.21.15.123:0/1080501856 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc800e070 con 0x7fabd0158f70 2023-10-06T04:31:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd558b700 1 -- 172.21.15.123:0/1080501856 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabc800bc30 con 0x7fabd0158f70 2023-10-06T04:31:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.858+0000 7fabd558b700 1 -- 172.21.15.123:0/1080501856 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabc8012430 con 0x7fabd0158f70 2023-10-06T04:31:47.863 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.859+0000 7fabd87f1700 1 -- 172.21.15.123:0/1080501856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0158f70 msgr2=0x7fabd01593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.859+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1080501856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0158f70 0x7fabd01593f0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fabc8009e60 tx=0x7fabc8009780 comp rx=0 tx=0).stop 2023-10-06T04:31:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.859+0000 7fabd87f1700 1 -- 172.21.15.123:0/1080501856 shutdown_connections 2023-10-06T04:31:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.859+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1080501856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0158f70 0x7fabd01593f0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.859+0000 7fabd87f1700 1 -- 172.21.15.123:0/1080501856 >> 172.21.15.123:0/1080501856 conn(0x7fabd00c9c90 msgr2=0x7fabd006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.859+0000 7fabd87f1700 1 -- 172.21.15.123:0/1080501856 shutdown_connections 2023-10-06T04:31:47.864 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.859+0000 7fabd87f1700 1 -- 172.21.15.123:0/1080501856 wait complete. 2023-10-06T04:31:47.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.860+0000 7fabd87f1700 1 Processor -- start 2023-10-06T04:31:47.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.860+0000 7fabd87f1700 1 -- start start 2023-10-06T04:31:47.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd00688c0 0x7fabd00c6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0068c90 0x7fabd00b0330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd0158f70 0x7fabd016ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015b7b0 con 0x7fabd0068c90 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd00b1370 con 0x7fabd0158f70 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd015b9e0 con 0x7fabd00688c0 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0068c90 0x7fabd00b0330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0068c90 0x7fabd00b0330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43632/0 (socket says 172.21.15.123:43632) 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 -- 172.21.15.123:0/1125283051 learned_addr learned my addr 172.21.15.123:0/1125283051 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd6d8e700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd0158f70 0x7fabd016ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd658d700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd00688c0 0x7fabd00c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 -- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd00688c0 msgr2=0x7fabd00c6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd00688c0 0x7fabd00c6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 -- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd0158f70 msgr2=0x7fabd016ca40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd0158f70 0x7fabd016ca40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 -- 172.21.15.123:0/1125283051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8009a20 con 0x7fabd0068c90 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd6d8e700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd0158f70 0x7fabd016ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd658d700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd00688c0 0x7fabd00c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd5d8c700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0068c90 0x7fabd00b0330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c7e0 tx=0x7fabcc007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabc67fc700 1 -- 172.21.15.123:0/1125283051 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc005b30 con 0x7fabd0068c90 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabc67fc700 1 -- 172.21.15.123:0/1125283051 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabcc005c90 con 0x7fabd0068c90 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd00b0db0 con 0x7fabd0068c90 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabc67fc700 1 -- 172.21.15.123:0/1125283051 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabcc007300 con 0x7fabd0068c90 2023-10-06T04:31:47.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.861+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd015bd00 con 0x7fabd0068c90 2023-10-06T04:31:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.862+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd00c6bb0 con 0x7fabd0068c90 2023-10-06T04:31:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.863+0000 7fabc67fc700 1 -- 172.21.15.123:0/1125283051 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fabcc01b400 con 0x7fabd0068c90 2023-10-06T04:31:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.863+0000 7fabc67fc700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabb4066f30 0x7fabb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.863+0000 7fabc67fc700 1 -- 172.21.15.123:0/1125283051 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fabcc0841b0 con 0x7fabd0068c90 2023-10-06T04:31:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.863+0000 7fabd658d700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabb4066f30 0x7fabb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.863+0000 7fabd658d700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabb4066f30 0x7fabb40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fabc0001840 tx=0x7fabc0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:47.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:47.867+0000 7fabc67fc700 1 -- 172.21.15.123:0/1125283051 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fabcc051340 con 0x7fabd0068c90 2023-10-06T04:31:48.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.049+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabd015be90 con 0x7fabd0068c90 2023-10-06T04:31:48.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.050+0000 7fabc67fc700 1 -- 172.21.15.123:0/1125283051 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7fabcc054960 con 0x7fabd0068c90 2023-10-06T04:31:48.053 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:48.053 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:48.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabb4066f30 msgr2=0x7fabb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabb4066f30 0x7fabb40693e0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fabc0001840 tx=0x7fabc0009040 comp rx=0 tx=0).stop 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0068c90 msgr2=0x7fabd00b0330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0068c90 0x7fabd00b0330 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7fabcc00c7e0 tx=0x7fabcc007df0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 shutdown_connections 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fabb4066f30 0x7fabb40693e0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fabd0158f70 0x7fabd016ca40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fabd00688c0 0x7fabd00c6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 --2- 172.21.15.123:0/1125283051 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fabd0068c90 0x7fabd00b0330 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 >> 172.21.15.123:0/1125283051 conn(0x7fabd00c9c90 msgr2=0x7fabd006d930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 shutdown_connections 2023-10-06T04:31:48.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.052+0000 7fabd87f1700 1 -- 172.21.15.123:0/1125283051 wait complete. 2023-10-06T04:31:48.067 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:31:32.045143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:31:33.390421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:31:48.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:31:48.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.185+0000 7f1cd9a65700 1 Processor -- start 2023-10-06T04:31:48.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd9a65700 1 -- start start 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd9a65700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40688c0 0x7f1cd40ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd9a65700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cd4068c90 0x7f1cd40abc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd9a65700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cd40c6de0 con 0x7f1cd40c7e70 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd9a65700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cd40ac190 con 0x7f1cd4068c90 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd9a65700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40c70c0 con 0x7f1cd40688c0 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1ccbfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1cd4068c90 0x7f1cd40abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd3fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1cd40c7e70 0x7f1cd40c82f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57082/0 (socket says 172.21.15.123:57082) 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.186+0000 7f1cd3fff700 1 -- 172.21.15.123:0/2975710293 learned_addr learned my addr 172.21.15.123:0/2975710293 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1cd2ffd700 1 -- 172.21.15.123:0/2975710293 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1687789080 0 0) 0x7f1cd40c6de0 con 0x7f1cd40c7e70 2023-10-06T04:31:48.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1cd2ffd700 1 -- 172.21.15.123:0/2975710293 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1cc0003570 con 0x7f1cd40c7e70 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1cd37fe700 1 --2- 172.21.15.123:0/2975710293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40688c0 0x7f1cd40ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1ccbfff700 1 -- 172.21.15.123:0/2975710293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40688c0 msgr2=0x7f1cd40ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1ccbfff700 1 --2- 172.21.15.123:0/2975710293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40688c0 0x7f1cd40ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1ccbfff700 1 -- 172.21.15.123:0/2975710293 >> v1:172.21.15.123:6789/0 conn(0x7f1cd40c7e70 legacy=0x7f1cd40c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1ccbfff700 1 -- 172.21.15.123:0/2975710293 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cd40c73a0 con 0x7f1cd4068c90 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1ccbfff700 1 --2- 172.21.15.123:0/2975710293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd4068c90 0x7f1cd40abc50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0098e0 tx=0x7f1cbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50ab19bedfd85b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1cd2ffd700 1 -- 172.21.15.123:0/2975710293 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00b640 con 0x7f1cd4068c90 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1cd2ffd700 1 -- 172.21.15.123:0/2975710293 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cbc00b7a0 con 0x7f1cd4068c90 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.187+0000 7f1cd2ffd700 1 -- 172.21.15.123:0/2975710293 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cbc00bb40 con 0x7f1cd4068c90 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2975710293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd4068c90 msgr2=0x7f1cd40abc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2975710293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd4068c90 0x7f1cd40abc50 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f1cbc0098e0 tx=0x7f1cbc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2975710293 shutdown_connections 2023-10-06T04:31:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2975710293 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40688c0 0x7f1cd40ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2975710293 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd4068c90 0x7f1cd40abc50 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2975710293 >> 172.21.15.123:0/2975710293 conn(0x7f1cd40c9ca0 msgr2=0x7f1cd406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2975710293 shutdown_connections 2023-10-06T04:31:48.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.188+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2975710293 wait complete. 2023-10-06T04:31:48.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.189+0000 7f1cd9a65700 1 Processor -- start 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.189+0000 7f1cd9a65700 1 -- start start 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd40688c0 0x7f1cd40c0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd4068c90 0x7f1cd40c5090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40c7e70 0x7f1cd4151a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd4165c00 con 0x7f1cd40688c0 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40c11f0 con 0x7f1cd4068c90 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cd40c14d0 con 0x7f1cd40c7e70 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd40688c0 0x7f1cd40c0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd40688c0 0x7f1cd40c0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43650/0 (socket says 172.21.15.123:43650) 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 -- 172.21.15.123:0/2555787046 learned_addr learned my addr 172.21.15.123:0/2555787046 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:48.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 -- 172.21.15.123:0/2555787046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40c7e70 msgr2=0x7f1cd4151a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40c7e70 0x7f1cd4151a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 -- 172.21.15.123:0/2555787046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd4068c90 msgr2=0x7f1cd40c5090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd4068c90 0x7f1cd40c5090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 -- 172.21.15.123:0/2555787046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc009420 con 0x7f1cd40688c0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd37fe700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd40688c0 0x7f1cd40c0c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400d890 tx=0x7f1cc4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1ccb7fe700 1 -- 172.21.15.123:0/2555787046 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400b600 con 0x7f1cd40688c0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1ccb7fe700 1 -- 172.21.15.123:0/2555787046 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cc400b760 con 0x7f1cd40688c0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1ccb7fe700 1 -- 172.21.15.123:0/2555787046 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cc400bae0 con 0x7f1cd40688c0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cd40c17b0 con 0x7f1cd40688c0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.190+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cd40c4980 con 0x7f1cd40688c0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.191+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cd4165ee0 con 0x7f1cd40688c0 2023-10-06T04:31:48.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.192+0000 7f1ccb7fe700 1 -- 172.21.15.123:0/2555787046 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1cc401e460 con 0x7f1cd40688c0 2023-10-06T04:31:48.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.192+0000 7f1ccb7fe700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1cb4066f30 0x7f1cb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.192+0000 7f1ccbfff700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1cb4066f30 0x7f1cb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.192+0000 7f1ccb7fe700 1 -- 172.21.15.123:0/2555787046 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1cc4084cb0 con 0x7f1cd40688c0 2023-10-06T04:31:48.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.193+0000 7f1ccbfff700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1cb4066f30 0x7f1cb40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40cb640 tx=0x7f1cbc017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.196+0000 7f1ccb7fe700 1 -- 172.21.15.123:0/2555787046 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1cc4051dc0 con 0x7f1cd40688c0 2023-10-06T04:31:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.380+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1cd40c7680 con 0x7f1cd40688c0 2023-10-06T04:31:48.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.381+0000 7f1ccb7fe700 1 -- 172.21.15.123:0/2555787046 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1cc40553e0 con 0x7f1cd40688c0 2023-10-06T04:31:48.383 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:31:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1cb4066f30 msgr2=0x7f1cb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1cb4066f30 0x7f1cb40693e0 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f1cd40cb640 tx=0x7f1cbc017000 comp rx=0 tx=0).stop 2023-10-06T04:31:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd40688c0 msgr2=0x7f1cd40c0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd40688c0 0x7f1cd40c0c10 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1cc400d890 tx=0x7f1cc4008040 comp rx=0 tx=0).stop 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 shutdown_connections 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1cb4066f30 0x7f1cb40693e0 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1cd40688c0 0x7f1cd40c0c10 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1cd40c7e70 0x7f1cd4151a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 --2- 172.21.15.123:0/2555787046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1cd4068c90 0x7f1cd40c5090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 >> 172.21.15.123:0/2555787046 conn(0x7f1cd40c9ca0 msgr2=0x7f1cd406ace0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 shutdown_connections 2023-10-06T04:31:48.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.383+0000 7f1cd9a65700 1 -- 172.21.15.123:0/2555787046 wait complete. 2023-10-06T04:31:48.397 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:31:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.515+0000 7fca68962700 1 Processor -- start 2023-10-06T04:31:48.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.515+0000 7fca68962700 1 -- start start 2023-10-06T04:31:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca68962700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca640a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca68962700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fca640c8710 0x7fca640c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca68962700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca64068670 0x7fca64068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca68962700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca64069050 con 0x7fca64068670 2023-10-06T04:31:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca68962700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca64069aa0 con 0x7fca640c8710 2023-10-06T04:31:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca68962700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca6406a340 con 0x7fca640c8340 2023-10-06T04:31:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca62d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca64068670 0x7fca64068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca62d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fca64068670 0x7fca64068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43664/0 (socket says 172.21.15.123:43664) 2023-10-06T04:31:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca62d9d700 1 -- 172.21.15.123:0/879843859 learned_addr learned my addr 172.21.15.123:0/879843859 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:48.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca61d9b700 1 --2- 172.21.15.123:0/879843859 >> v2:172.21.15.177:3300/0 conn(0x7fca640c8710 0x7fca640c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.516+0000 7fca6259c700 1 --2- 172.21.15.123:0/879843859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca640a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca62d9d700 1 -- 172.21.15.123:0/879843859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 msgr2=0x7fca640a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca62d9d700 1 --2- 172.21.15.123:0/879843859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca640a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca62d9d700 1 -- 172.21.15.123:0/879843859 >> v2:172.21.15.177:3300/0 conn(0x7fca640c8710 msgr2=0x7fca640c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca62d9d700 1 --2- 172.21.15.123:0/879843859 >> v2:172.21.15.177:3300/0 conn(0x7fca640c8710 0x7fca640c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca62d9d700 1 -- 172.21.15.123:0/879843859 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca640aa6f0 con 0x7fca64068670 2023-10-06T04:31:48.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca61d9b700 1 --2- 172.21.15.123:0/879843859 >> v2:172.21.15.177:3300/0 conn(0x7fca640c8710 0x7fca640c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca62d9d700 1 --2- 172.21.15.123:0/879843859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca64068b10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009f20 tx=0x7fca5c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aee158afddab53b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca6159a700 1 -- 172.21.15.123:0/879843859 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c00e070 con 0x7fca64068670 2023-10-06T04:31:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca6159a700 1 -- 172.21.15.123:0/879843859 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5c00ba40 con 0x7fca64068670 2023-10-06T04:31:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca6159a700 1 -- 172.21.15.123:0/879843859 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5c0092e0 con 0x7fca64068670 2023-10-06T04:31:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 -- 172.21.15.123:0/879843859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 msgr2=0x7fca64068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 --2- 172.21.15.123:0/879843859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca64068b10 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009f20 tx=0x7fca5c009840 comp rx=0 tx=0).stop 2023-10-06T04:31:48.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 -- 172.21.15.123:0/879843859 shutdown_connections 2023-10-06T04:31:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 --2- 172.21.15.123:0/879843859 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca64068b10 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 --2- 172.21.15.123:0/879843859 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca640a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 --2- 172.21.15.123:0/879843859 >> v2:172.21.15.177:3300/0 conn(0x7fca640c8710 0x7fca640c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 -- 172.21.15.123:0/879843859 >> 172.21.15.123:0/879843859 conn(0x7fca640c9ca0 msgr2=0x7fca6406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.517+0000 7fca68962700 1 -- 172.21.15.123:0/879843859 shutdown_connections 2023-10-06T04:31:48.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.518+0000 7fca68962700 1 -- 172.21.15.123:0/879843859 wait complete. 2023-10-06T04:31:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 Processor -- start 2023-10-06T04:31:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 -- start start 2023-10-06T04:31:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca640c72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca6415a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca640c8710 0x7fca640c4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640c6e20 con 0x7fca64068670 2023-10-06T04:31:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca640c6530 con 0x7fca640c8710 2023-10-06T04:31:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca68962700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca640c4de0 con 0x7fca640c8340 2023-10-06T04:31:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca6259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca640c72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca6259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca640c72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43674/0 (socket says 172.21.15.123:43674) 2023-10-06T04:31:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.519+0000 7fca6259c700 1 -- 172.21.15.123:0/2254440667 learned_addr learned my addr 172.21.15.123:0/2254440667 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca62d9d700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca640c8710 0x7fca640c4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca61d9b700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca6415a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca6259c700 1 -- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 msgr2=0x7fca6415a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca6259c700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca6415a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca6259c700 1 -- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca640c8710 msgr2=0x7fca640c4520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca6259c700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca640c8710 0x7fca640c4520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca6259c700 1 -- 172.21.15.123:0/2254440667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca5c009ae0 con 0x7fca64068670 2023-10-06T04:31:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca6259c700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca640c72b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d890 tx=0x7fca58008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca527fc700 1 -- 172.21.15.123:0/2254440667 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800b600 con 0x7fca64068670 2023-10-06T04:31:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca527fc700 1 -- 172.21.15.123:0/2254440667 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca5800b760 con 0x7fca64068670 2023-10-06T04:31:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca640c67d0 con 0x7fca64068670 2023-10-06T04:31:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca527fc700 1 -- 172.21.15.123:0/2254440667 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca5800bae0 con 0x7fca64068670 2023-10-06T04:31:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.520+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca640c4a60 con 0x7fca64068670 2023-10-06T04:31:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.521+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca64069cf0 con 0x7fca64068670 2023-10-06T04:31:48.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.521+0000 7fca527fc700 1 -- 172.21.15.123:0/2254440667 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fca5801e460 con 0x7fca64068670 2023-10-06T04:31:48.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.522+0000 7fca527fc700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fca44066f30 0x7fca440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.522+0000 7fca527fc700 1 -- 172.21.15.123:0/2254440667 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fca58084d40 con 0x7fca64068670 2023-10-06T04:31:48.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.522+0000 7fca61d9b700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fca44066f30 0x7fca440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.522+0000 7fca61d9b700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fca44066f30 0x7fca440693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca54006f00 tx=0x7fca54008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.526+0000 7fca527fc700 1 -- 172.21.15.123:0/2254440667 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fca58051e50 con 0x7fca64068670 2023-10-06T04:31:48.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.706+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fca640c7100 con 0x7fca64068670 2023-10-06T04:31:48.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.706+0000 7fca527fc700 1 -- 172.21.15.123:0/2254440667 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fca58055470 con 0x7fca64068670 2023-10-06T04:31:48.709 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:31:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fca44066f30 msgr2=0x7fca440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fca44066f30 0x7fca440693e0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fca54006f00 tx=0x7fca54008040 comp rx=0 tx=0).stop 2023-10-06T04:31:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 msgr2=0x7fca640c72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca640c72b0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca5800d890 tx=0x7fca58008040 comp rx=0 tx=0).stop 2023-10-06T04:31:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 shutdown_connections 2023-10-06T04:31:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fca44066f30 0x7fca440693e0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fca64068670 0x7fca640c72b0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fca640c8710 0x7fca640c4520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 --2- 172.21.15.123:0/2254440667 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fca640c8340 0x7fca6415a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 >> 172.21.15.123:0/2254440667 conn(0x7fca640c9ca0 msgr2=0x7fca64158bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:48.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 shutdown_connections 2023-10-06T04:31:48.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.708+0000 7fca68962700 1 -- 172.21.15.123:0/2254440667 wait complete. 2023-10-06T04:31:48.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:31:48.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.840+0000 7f29c3ec3700 1 Processor -- start 2023-10-06T04:31:48.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.840+0000 7f29c3ec3700 1 -- start start 2023-10-06T04:31:48.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c3ec3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 0x7f29bc0a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c3ec3700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f29bc068d80 0x7f29bc0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c3ec3700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29bc069aa0 con 0x7f29bc0c8710 2023-10-06T04:31:48.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c3ec3700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29bc06a340 con 0x7f29bc068d80 2023-10-06T04:31:48.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c3ec3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0aa6f0 con 0x7f29bc0c8340 2023-10-06T04:31:48.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c145e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f29bc0c8710 0x7f29bc068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57096/0 (socket says 172.21.15.123:57096) 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c145e700 1 -- 172.21.15.123:0/1760044509 learned_addr learned my addr 172.21.15.123:0/1760044509 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c1c5f700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 0x7f29bc0a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c2460700 1 --2- 172.21.15.123:0/1760044509 >> v2:172.21.15.177:3300/0 conn(0x7f29bc068d80 0x7f29bc0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2096224818 0 0) 0x7f29bc069aa0 con 0x7f29bc0c8710 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f29a8003570 con 0x7f29bc0c8710 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3745567561 0 0) 0x7f29a8003570 con 0x7f29bc0c8710 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 msgr2=0x7f29bc0a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:48.848 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 0x7f29bc0a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 >> v2:172.21.15.177:3300/0 conn(0x7f29bc068d80 msgr2=0x7f29bc0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 --2- 172.21.15.123:0/1760044509 >> v2:172.21.15.177:3300/0 conn(0x7f29bc068d80 0x7f29bc0c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc069cf0 con 0x7f29bc0c8710 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f29b8001470 con 0x7f29bc0c8710 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 >> v1:172.21.15.123:6789/0 conn(0x7f29bc0c8710 legacy=0x7f29bc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 0x7f29a8003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29a8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a8004d50 0x7f29a80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc06a340 con 0x7f29a8003fe0 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0aa6f0 con 0x7f29a8004d50 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29a800ba90 con 0x7f29bc0c8340 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.841+0000 7f29c145e700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29a8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c1c5f700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 0x7f29a8003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c2460700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a8004d50 0x7f29a80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c145e700 1 -- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 msgr2=0x7f29a8003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:48.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c145e700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c8340 0x7f29a8003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c145e700 1 -- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a8004d50 msgr2=0x7f29a80051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c145e700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29a8004d50 0x7f29a80051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c145e700 1 -- 172.21.15.123:0/1760044509 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29a8003570 con 0x7f29a8003fe0 2023-10-06T04:31:48.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c145e700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29a8004810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c3c0 tx=0x7f29bc06a480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13dff03156d9693 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b800cc30 con 0x7f29a8003fe0 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29b800cdd0 con 0x7f29a8003fe0 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c0c5d700 1 -- 172.21.15.123:0/1760044509 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29b8005b70 con 0x7f29a8003fe0 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c3ec3700 1 -- 172.21.15.123:0/1760044509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 msgr2=0x7f29a8004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c3ec3700 1 --2- 172.21.15.123:0/1760044509 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29a8004810 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c3c0 tx=0x7f29bc06a480 comp rx=0 tx=0).stop 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c2460700 1 -- 172.21.15.123:0/1760044509 reap_dead start 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c3ec3700 1 -- 172.21.15.123:0/1760044509 shutdown_connections 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.842+0000 7f29c3ec3700 1 -- 172.21.15.123:0/1760044509 >> 172.21.15.123:0/1760044509 conn(0x7f29bc0c9ca0 msgr2=0x7f29bc06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:48.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.843+0000 7f29c3ec3700 1 -- 172.21.15.123:0/1760044509 shutdown_connections 2023-10-06T04:31:48.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.843+0000 7f29c3ec3700 1 -- 172.21.15.123:0/1760044509 wait complete. 2023-10-06T04:31:48.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.844+0000 7f29c3ec3700 1 Processor -- start 2023-10-06T04:31:48.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.844+0000 7f29c3ec3700 1 -- start start 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c3ec3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29bc16d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c3ec3700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c5310 0x7f29bc06aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c3ec3700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc06b3e0 0x7f29bc0abe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c3ec3700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc06b880 con 0x7f29a8003fe0 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c3ec3700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc16deb0 con 0x7f29bc06b3e0 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c3ec3700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29bc0ae190 con 0x7f29bc0c5310 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29bc16d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29bc16d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43692/0 (socket says 172.21.15.123:43692) 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 -- 172.21.15.123:0/3794414903 learned_addr learned my addr 172.21.15.123:0/3794414903 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c2460700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc06b3e0 0x7f29bc0abe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c145e700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c5310 0x7f29bc06aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 -- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c5310 msgr2=0x7f29bc06aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c5310 0x7f29bc06aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 -- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc06b3e0 msgr2=0x7f29bc0abe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc06b3e0 0x7f29bc0abe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.845+0000 7f29c1c5f700 1 -- 172.21.15.123:0/3794414903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29bc0ae470 con 0x7f29a8003fe0 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.846+0000 7f29c1c5f700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29bc16d970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29ac011d40 tx=0x7f29ac0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.846+0000 7f29b1ffb700 1 -- 172.21.15.123:0/3794414903 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00f930 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.846+0000 7f29b1ffb700 1 -- 172.21.15.123:0/3794414903 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29ac00fa90 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.846+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29bc0ae750 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.846+0000 7f29b1ffb700 1 -- 172.21.15.123:0/3794414903 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29ac00fe10 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.846+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29bc0aed10 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.847+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29bc069fd0 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.847+0000 7f29b1ffb700 1 -- 172.21.15.123:0/3794414903 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f29ac014650 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.848+0000 7f29b1ffb700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f29a0066fb0 0x7f29a0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.848+0000 7f29b1ffb700 1 -- 172.21.15.123:0/3794414903 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f29ac018030 con 0x7f29a8003fe0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.848+0000 7f29c145e700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f29a0066fb0 0x7f29a0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:48.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.848+0000 7f29c145e700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f29a0066fb0 0x7f29a0069460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c3c0 tx=0x7f29b8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:48.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:48.852+0000 7f29b1ffb700 1 -- 172.21.15.123:0/3794414903 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f29ac052060 con 0x7f29a8003fe0 2023-10-06T04:31:49.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.033+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f29bc04bb80 con 0x7f29a8003fe0 2023-10-06T04:31:49.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.034+0000 7f29b1ffb700 1 -- 172.21.15.123:0/3794414903 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f29ac022020 con 0x7f29a8003fe0 2023-10-06T04:31:49.036 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:31:49.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f29a0066fb0 msgr2=0x7f29a0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f29a0066fb0 0x7f29a0069460 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7f29b800c3c0 tx=0x7f29b8015000 comp rx=0 tx=0).stop 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 msgr2=0x7f29bc16d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29bc16d970 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f29ac011d40 tx=0x7f29ac0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 shutdown_connections 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f29a0066fb0 0x7f29a0069460 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f29a8003fe0 0x7f29bc16d970 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f29bc06b3e0 0x7f29bc0abe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 --2- 172.21.15.123:0/3794414903 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f29bc0c5310 0x7f29bc06aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 >> 172.21.15.123:0/3794414903 conn(0x7f29bc0c9ca0 msgr2=0x7f29bc0cbf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:49.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 shutdown_connections 2023-10-06T04:31:49.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.036+0000 7f29c3ec3700 1 -- 172.21.15.123:0/3794414903 wait complete. 2023-10-06T04:31:49.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:31:49.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.168+0000 7efd3036b700 1 Processor -- start 2023-10-06T04:31:49.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.168+0000 7efd3036b700 1 -- start start 2023-10-06T04:31:49.172 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd3036b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28154ee0 0x7efd280c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.172 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd3036b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efd28158670 0x7efd28067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd3036b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd28068000 con 0x7efd28067ba0 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd3036b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efd2806ecb0 con 0x7efd28158670 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd3036b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd2806b3c0 con 0x7efd28154ee0 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2e107700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28154ee0 0x7efd280c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2d906700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efd28158670 0x7efd28067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2e908700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7efd28067ba0 0x7efd2815afa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:53490/0 (socket says 172.21.15.123:53490) 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2e908700 1 -- 172.21.15.123:0/1507543631 learned_addr learned my addr 172.21.15.123:0/1507543631 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2e107700 1 --2- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28154ee0 0x7efd280c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:46858/0 (socket says 172.21.15.123:46858) 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2d906700 1 --2- 172.21.15.123:0/1507543631 >> v2:172.21.15.177:3300/0 conn(0x7efd28158670 0x7efd28067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:41506/0 (socket says 172.21.15.123:41506) 2023-10-06T04:31:49.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2d906700 1 -- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28154ee0 msgr2=0x7efd280c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:49.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2d906700 1 --2- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28154ee0 0x7efd280c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2d906700 1 -- 172.21.15.123:0/1507543631 >> v1:172.21.15.177:6789/0 conn(0x7efd28067ba0 legacy=0x7efd2815afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2d906700 1 -- 172.21.15.123:0/1507543631 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd280c7f90 con 0x7efd28158670 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.169+0000 7efd2d906700 1 --2- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28158670 0x7efd28067660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd180098e0 tx=0x7efd180091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=27e08a57edb3b1a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd2d105700 1 -- 172.21.15.123:0/1507543631 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800b640 con 0x7efd28158670 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd2d105700 1 -- 172.21.15.123:0/1507543631 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd1800b7a0 con 0x7efd28158670 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd2d105700 1 -- 172.21.15.123:0/1507543631 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd1800bb40 con 0x7efd28158670 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 -- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28158670 msgr2=0x7efd28067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 --2- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28158670 0x7efd28067660 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd180098e0 tx=0x7efd180091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 -- 172.21.15.123:0/1507543631 shutdown_connections 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 --2- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28154ee0 0x7efd280c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 --2- 172.21.15.123:0/1507543631 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28158670 0x7efd28067660 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 -- 172.21.15.123:0/1507543631 >> 172.21.15.123:0/1507543631 conn(0x7efd280cd020 msgr2=0x7efd280cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:49.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 -- 172.21.15.123:0/1507543631 shutdown_connections 2023-10-06T04:31:49.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.170+0000 7efd3036b700 1 -- 172.21.15.123:0/1507543631 wait complete. 2023-10-06T04:31:49.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.171+0000 7efd3036b700 1 Processor -- start 2023-10-06T04:31:49.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.171+0000 7efd3036b700 1 -- start start 2023-10-06T04:31:49.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd3036b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd28067ba0 0x7efd280cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd3036b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28154ee0 0x7efd280af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd3036b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28158670 0x7efd280b1df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd3036b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280aee50 con 0x7efd28067ba0 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd3036b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd280cae20 con 0x7efd28154ee0 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd3036b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd280b2330 con 0x7efd28158670 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd28067ba0 0x7efd280cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd28067ba0 0x7efd280cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43696/0 (socket says 172.21.15.123:43696) 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 -- 172.21.15.123:0/3250462684 learned_addr learned my addr 172.21.15.123:0/3250462684 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2d906700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28154ee0 0x7efd280af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e908700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28158670 0x7efd280b1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 -- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28158670 msgr2=0x7efd280b1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28158670 0x7efd280b1df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 -- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28154ee0 msgr2=0x7efd280af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28154ee0 0x7efd280af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.172+0000 7efd2e107700 1 -- 172.21.15.123:0/3250462684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd18009420 con 0x7efd28067ba0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.173+0000 7efd2e107700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd28067ba0 0x7efd280cbe30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e930 tx=0x7efd240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.173+0000 7efd1e7fc700 1 -- 172.21.15.123:0/3250462684 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24013680 con 0x7efd28067ba0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.173+0000 7efd1e7fc700 1 -- 172.21.15.123:0/3250462684 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd240137e0 con 0x7efd28067ba0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.173+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd280cc410 con 0x7efd28067ba0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.173+0000 7efd1e7fc700 1 -- 172.21.15.123:0/3250462684 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd24013b80 con 0x7efd28067ba0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.173+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd280cc9d0 con 0x7efd28067ba0 2023-10-06T04:31:49.180 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.174+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd280c25a0 con 0x7efd28067ba0 2023-10-06T04:31:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.174+0000 7efd1e7fc700 1 -- 172.21.15.123:0/3250462684 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd2400c3e0 con 0x7efd28067ba0 2023-10-06T04:31:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.174+0000 7efd1e7fc700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd0c066f30 0x7efd0c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.175+0000 7efd1e7fc700 1 -- 172.21.15.123:0/3250462684 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efd24084700 con 0x7efd28067ba0 2023-10-06T04:31:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.175+0000 7efd2d906700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd0c066f30 0x7efd0c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.175+0000 7efd2d906700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd0c066f30 0x7efd0c0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd28158aa0 tx=0x7efd18017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.179+0000 7efd1e7fc700 1 -- 172.21.15.123:0/3250462684 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd24051810 con 0x7efd28067ba0 2023-10-06T04:31:49.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.361+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7efd280cabf0 con 0x7efd28067ba0 2023-10-06T04:31:49.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.361+0000 7efd1e7fc700 1 -- 172.21.15.123:0/3250462684 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7efd24054e30 con 0x7efd28067ba0 2023-10-06T04:31:49.364 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:31:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd0c066f30 msgr2=0x7efd0c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd0c066f30 0x7efd0c0693e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7efd28158aa0 tx=0x7efd18017000 comp rx=0 tx=0).stop 2023-10-06T04:31:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd28067ba0 msgr2=0x7efd280cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd28067ba0 0x7efd280cbe30 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7efd2400e930 tx=0x7efd240091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 shutdown_connections 2023-10-06T04:31:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd0c066f30 0x7efd0c0693e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd28067ba0 0x7efd280cbe30 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd28158670 0x7efd280b1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 --2- 172.21.15.123:0/3250462684 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd28154ee0 0x7efd280af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 >> 172.21.15.123:0/3250462684 conn(0x7efd280cd020 msgr2=0x7efd280c1e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 shutdown_connections 2023-10-06T04:31:49.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.363+0000 7efd3036b700 1 -- 172.21.15.123:0/3250462684 wait complete. 2023-10-06T04:31:49.377 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:31:49.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.492+0000 7fcc6a98f700 1 Processor -- start 2023-10-06T04:31:49.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.492+0000 7fcc6a98f700 1 -- start start 2023-10-06T04:31:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc6a98f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc640688c0 0x7fcc6406a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc6a98f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc64068c90 0x7fcc641574f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc6a98f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc640cc190 con 0x7fcc6406a930 2023-10-06T04:31:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc6a98f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc640cc620 con 0x7fcc640688c0 2023-10-06T04:31:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc6a98f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640cb370 con 0x7fcc64068c90 2023-10-06T04:31:49.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc63fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcc640688c0 0x7fcc6406a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc637fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcc6406a930 0x7fcc64156de0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57112/0 (socket says 172.21.15.123:57112) 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc637fe700 1 -- 172.21.15.123:0/4091285328 learned_addr learned my addr 172.21.15.123:0/4091285328 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc68f2c700 1 --2- 172.21.15.123:0/4091285328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc64068c90 0x7fcc641574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc63fff700 1 -- 172.21.15.123:0/4091285328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc64068c90 msgr2=0x7fcc641574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc63fff700 1 --2- 172.21.15.123:0/4091285328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc64068c90 0x7fcc641574f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc63fff700 1 -- 172.21.15.123:0/4091285328 >> v1:172.21.15.123:6789/0 conn(0x7fcc6406a930 legacy=0x7fcc64156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc63fff700 1 -- 172.21.15.123:0/4091285328 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc640cb580 con 0x7fcc640688c0 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc62ffd700 1 -- 172.21.15.123:0/4091285328 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067418550 0 0) 0x7fcc640cc190 con 0x7fcc6406a930 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.493+0000 7fcc63fff700 1 --2- 172.21.15.123:0/4091285328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc640688c0 0x7fcc6406a3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc580098e0 tx=0x7fcc580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94178a445a9168c0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc62ffd700 1 -- 172.21.15.123:0/4091285328 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc58009e80 con 0x7fcc640688c0 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc62ffd700 1 -- 172.21.15.123:0/4091285328 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc5800b860 con 0x7fcc640688c0 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc62ffd700 1 -- 172.21.15.123:0/4091285328 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc5800bbe0 con 0x7fcc640688c0 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 -- 172.21.15.123:0/4091285328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc640688c0 msgr2=0x7fcc6406a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/4091285328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc640688c0 0x7fcc6406a3f0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fcc580098e0 tx=0x7fcc580091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 -- 172.21.15.123:0/4091285328 shutdown_connections 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/4091285328 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc64068c90 0x7fcc641574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/4091285328 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc640688c0 0x7fcc6406a3f0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 -- 172.21.15.123:0/4091285328 >> 172.21.15.123:0/4091285328 conn(0x7fcc640c9c90 msgr2=0x7fcc6406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 -- 172.21.15.123:0/4091285328 shutdown_connections 2023-10-06T04:31:49.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.494+0000 7fcc6a98f700 1 -- 172.21.15.123:0/4091285328 wait complete. 2023-10-06T04:31:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.495+0000 7fcc6a98f700 1 Processor -- start 2023-10-06T04:31:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc6a98f700 1 -- start start 2023-10-06T04:31:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc6a98f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc640688c0 0x7fcc641535c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc6a98f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64068c90 0x7fcc640af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc6a98f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6406a930 0x7fcc640afbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc6a98f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc640b2150 con 0x7fcc64068c90 2023-10-06T04:31:49.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc6a98f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc641684f0 con 0x7fcc640688c0 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc6a98f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc64167bc0 con 0x7fcc6406a930 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64068c90 0x7fcc640af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc637fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64068c90 0x7fcc640af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43708/0 (socket says 172.21.15.123:43708) 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.496+0000 7fcc637fe700 1 -- 172.21.15.123:0/260215680 learned_addr learned my addr 172.21.15.123:0/260215680 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc68f2c700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6406a930 0x7fcc640afbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc637fe700 1 -- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6406a930 msgr2=0x7fcc640afbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc63fff700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc640688c0 0x7fcc641535c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc637fe700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6406a930 0x7fcc640afbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc637fe700 1 -- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc640688c0 msgr2=0x7fcc641535c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc637fe700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc640688c0 0x7fcc641535c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc637fe700 1 -- 172.21.15.123:0/260215680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc58009420 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc68f2c700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6406a930 0x7fcc640afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc637fe700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64068c90 0x7fcc640af6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400c7e0 tx=0x7fcc54007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc4bfff700 1 -- 172.21.15.123:0/260215680 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54005b30 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc4bfff700 1 -- 172.21.15.123:0/260215680 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc54005c90 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc64167ea0 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc4bfff700 1 -- 172.21.15.123:0/260215680 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc54007300 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.497+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc64153b00 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.498+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc64168680 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.498+0000 7fcc4bfff700 1 -- 172.21.15.123:0/260215680 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcc5401b400 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.499+0000 7fcc4bfff700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcc50066f30 0x7fcc500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.499+0000 7fcc4bfff700 1 -- 172.21.15.123:0/260215680 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcc54084190 con 0x7fcc64068c90 2023-10-06T04:31:49.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.499+0000 7fcc63fff700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcc50066f30 0x7fcc500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.499+0000 7fcc63fff700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcc50066f30 0x7fcc500693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc640b09d0 tx=0x7fcc58017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.503+0000 7fcc4bfff700 1 -- 172.21.15.123:0/260215680 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcc54051320 con 0x7fcc64068c90 2023-10-06T04:31:49.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.727+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc640cb860 con 0x7fcc64068c90 2023-10-06T04:31:49.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.727+0000 7fcc4bfff700 1 -- 172.21.15.123:0/260215680 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7fcc54054940 con 0x7fcc64068c90 2023-10-06T04:31:49.729 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:49.730 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:31:49.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcc50066f30 msgr2=0x7fcc500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcc50066f30 0x7fcc500693e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fcc640b09d0 tx=0x7fcc58017000 comp rx=0 tx=0).stop 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64068c90 msgr2=0x7fcc640af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64068c90 0x7fcc640af6a0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fcc5400c7e0 tx=0x7fcc54007df0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 shutdown_connections 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcc50066f30 0x7fcc500693e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcc6406a930 0x7fcc640afbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcc640688c0 0x7fcc641535c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 --2- 172.21.15.123:0/260215680 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcc64068c90 0x7fcc640af6a0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 >> 172.21.15.123:0/260215680 conn(0x7fcc640c9c90 msgr2=0x7fcc640c5820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:49.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 shutdown_connections 2023-10-06T04:31:49.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.729+0000 7fcc6a98f700 1 -- 172.21.15.123:0/260215680 wait complete. 2023-10-06T04:31:49.733 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:31:49.744 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:31:49.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.858+0000 7f9bf1a1c700 1 Processor -- start 2023-10-06T04:31:49.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.859+0000 7f9bf1a1c700 1 -- start start 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.859+0000 7f9bf1a1c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 0x7f9bec0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.859+0000 7f9bf1a1c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bec15b0d0 con 0x7f9bec0b05b0 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.859+0000 7f9bf1a1c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bec15b370 con 0x7f9bec1591b0 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.859+0000 7f9bf1a1c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec159c90 con 0x7f9bec0b43b0 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.859+0000 7f9bea7fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9bec0b05b0 0x7f9bec0b3ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57120/0 (socket says 172.21.15.123:57120) 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.859+0000 7f9bea7fc700 1 -- 172.21.15.123:0/1019095766 learned_addr learned my addr 172.21.15.123:0/1019095766 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 0x7f9bec0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3448564930 0 0) 0x7f9bec15b0d0 con 0x7f9bec0b05b0 2023-10-06T04:31:49.865 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd4003570 con 0x7f9bec0b05b0 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3703728744 0 0) 0x7f9bec15b370 con 0x7f9bec1591b0 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bec15b0d0 con 0x7f9bec1591b0 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2448340098 0 0) 0x7f9bd4003570 con 0x7f9bec0b05b0 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 msgr2=0x7f9bec0cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 0x7f9bec0cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 >> v1:172.21.15.177:6789/0 conn(0x7f9bec1591b0 legacy=0x7f9bec159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec0b0fe0 con 0x7f9bec0b05b0 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f9bdc001470 con 0x7f9bec0b05b0 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 >> v1:172.21.15.123:6789/0 conn(0x7f9bec0b05b0 legacy=0x7f9bec0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 0x7f9bd4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004180 0x7f9bd4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.866 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bd4004d50 0x7f9bd40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd400bb70 con 0x7f9bd4004d50 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd400bce0 con 0x7f9bd4004180 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd400bea0 con 0x7f9bec0b43b0 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bd4004d50 0x7f9bd40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9bea7fc700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004180 0x7f9bd4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beaffd700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 0x7f9bd4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 -- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 msgr2=0x7f9bd4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 0x7f9bd4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 -- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004180 msgr2=0x7f9bd4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004180 0x7f9bd4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 -- 172.21.15.123:0/1019095766 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd4003570 con 0x7f9bd4004d50 2023-10-06T04:31:49.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beaffd700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0b43b0 0x7f9bd4003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9bea7fc700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004180 0x7f9bd4004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.860+0000 7f9beb7fe700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bd4004d50 0x7f9bd40051c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9bec09e170 tx=0x7f9be400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1cc02f84a80c8e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be400ea60 con 0x7f9bd4004d50 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9be400f040 con 0x7f9bd4004d50 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9be9ffb700 1 -- 172.21.15.123:0/1019095766 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9be400ee10 con 0x7f9bd4004d50 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/1019095766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bd4004d50 msgr2=0x7f9bd40051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9bf1a1c700 1 --2- 172.21.15.123:0/1019095766 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bd4004d50 0x7f9bd40051c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9bec09e170 tx=0x7f9be400b040 comp rx=0 tx=0).stop 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9beb7fe700 1 -- 172.21.15.123:0/1019095766 reap_dead start 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/1019095766 shutdown_connections 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/1019095766 >> 172.21.15.123:0/1019095766 conn(0x7f9bec0995a0 msgr2=0x7f9bec063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/1019095766 shutdown_connections 2023-10-06T04:31:49.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.861+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/1019095766 wait complete. 2023-10-06T04:31:49.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.862+0000 7f9bf1a1c700 1 Processor -- start 2023-10-06T04:31:49.869 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.862+0000 7f9bf1a1c700 1 -- start start 2023-10-06T04:31:49.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bf1a1c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004d50 0x7f9bec157820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bf1a1c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bec09edf0 0x7f9bec0aeb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bf1a1c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0a0160 0x7f9bec0a05e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bf1a1c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0a0b20 con 0x7f9bec09edf0 2023-10-06T04:31:49.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bf1a1c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec0ac300 con 0x7f9bd4004d50 2023-10-06T04:31:49.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bf1a1c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bec157d60 con 0x7f9bec0a0160 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bea7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bec09edf0 0x7f9bec0aeb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bea7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bec09edf0 0x7f9bec0aeb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43724/0 (socket says 172.21.15.123:43724) 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9bea7fc700 1 -- 172.21.15.123:0/529187793 learned_addr learned my addr 172.21.15.123:0/529187793 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9beaffd700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004d50 0x7f9bec157820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.863+0000 7f9beb7fe700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0a0160 0x7f9bec0a05e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bea7fc700 1 -- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0a0160 msgr2=0x7f9bec0a05e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bea7fc700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0a0160 0x7f9bec0a05e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bea7fc700 1 -- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004d50 msgr2=0x7f9bec157820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bea7fc700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004d50 0x7f9bec157820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bea7fc700 1 -- 172.21.15.123:0/529187793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bec09f2b0 con 0x7f9bec09edf0 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9beaffd700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004d50 0x7f9bec157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:49.872 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bea7fc700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bec09edf0 0x7f9bec0aeb10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e850 tx=0x7f9bdc001dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bd2ffd700 1 -- 172.21.15.123:0/529187793 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc012500 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bd2ffd700 1 -- 172.21.15.123:0/529187793 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bdc010040 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bec09f590 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bd2ffd700 1 -- 172.21.15.123:0/529187793 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bdc012870 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.864+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bec09fb50 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.865+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bec0ac4b0 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.865+0000 7f9bd2ffd700 1 -- 172.21.15.123:0/529187793 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9bdc012c20 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.866+0000 7f9bd2ffd700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9bd8066f60 0x7f9bd8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.866+0000 7f9beaffd700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9bd8066f60 0x7f9bd8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.866+0000 7f9bd2ffd700 1 -- 172.21.15.123:0/529187793 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9bdc083890 con 0x7f9bec09edf0 2023-10-06T04:31:49.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.866+0000 7f9beaffd700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9bd8066f60 0x7f9bd8069410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000f60 tx=0x7f9be0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:49.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:49.870+0000 7f9bd2ffd700 1 -- 172.21.15.123:0/529187793 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9bdc04d5a0 con 0x7f9bec09edf0 2023-10-06T04:31:50.059 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.056+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9bec0b12c0 con 0x7f9bec09edf0 2023-10-06T04:31:50.059 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.057+0000 7f9bd2ffd700 1 -- 172.21.15.123:0/529187793 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f9bdc019070 con 0x7f9bec09edf0 2023-10-06T04:31:50.060 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:50.060 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:31:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9bd8066f60 msgr2=0x7f9bd8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9bd8066f60 0x7f9bd8069410 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f9be0000f60 tx=0x7f9be0009040 comp rx=0 tx=0).stop 2023-10-06T04:31:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bec09edf0 msgr2=0x7f9bec0aeb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bec09edf0 0x7f9bec0aeb10 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f9bdc00e850 tx=0x7f9bdc001dc0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.062 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 shutdown_connections 2023-10-06T04:31:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9bd8066f60 0x7f9bd8069410 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9bec0a0160 0x7f9bec0a05e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9bd4004d50 0x7f9bec157820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 --2- 172.21.15.123:0/529187793 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9bec09edf0 0x7f9bec0aeb10 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 >> 172.21.15.123:0/529187793 conn(0x7f9bec0995a0 msgr2=0x7f9bec069750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 shutdown_connections 2023-10-06T04:31:50.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.059+0000 7f9bf1a1c700 1 -- 172.21.15.123:0/529187793 wait complete. 2023-10-06T04:31:50.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:31:50.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.189+0000 7f0460dcb700 1 Processor -- start 2023-10-06T04:31:50.191 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.189+0000 7f0460dcb700 1 -- start start 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f0460dcb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 0x7f045c06a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f0460dcb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c0cc190 con 0x7f045c068c90 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f0460dcb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f045c0cb580 con 0x7f045c06a930 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f0460dcb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0cc620 con 0x7f045c0688c0 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f045ad9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f045c068c90 0x7f045c0c3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:57136/0 (socket says 172.21.15.123:57136) 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f045ad9d700 1 -- 172.21.15.123:0/4285222739 learned_addr learned my addr 172.21.15.123:0/4285222739 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f045a59c700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 0x7f045c06a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1630586315 0 0) 0x7f045c0cc190 con 0x7f045c068c90 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.190+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0444003570 con 0x7f045c068c90 2023-10-06T04:31:50.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 576613184 0 0) 0x7f0444003570 con 0x7f045c068c90 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 msgr2=0x7f045c06a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 0x7f045c06a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 >> v1:172.21.15.177:6789/0 conn(0x7f045c06a930 legacy=0x7f045c156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c0cb860 con 0x7f045c068c90 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f04540017f0 con 0x7f045c068c90 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 >> v1:172.21.15.123:6789/0 conn(0x7f045c068c90 legacy=0x7f045c0c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0444003a90 0x7f0444003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 0x7f0444004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0444004d50 0x7f04440051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0cc620 con 0x7f0444003a90 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044400bb10 con 0x7f0444004d50 2023-10-06T04:31:50.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f044400bcd0 con 0x7f045c0688c0 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045a59c700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0444003a90 0x7f0444003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045ad9d700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0444004d50 0x7f04440051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f0459d9b700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 0x7f0444004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045a59c700 1 -- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 msgr2=0x7f0444004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045a59c700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f045c0688c0 0x7f0444004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045a59c700 1 -- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0444004d50 msgr2=0x7f04440051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045a59c700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0444004d50 0x7f04440051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045a59c700 1 -- 172.21.15.123:0/4285222739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0444003570 con 0x7f0444003a90 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045ad9d700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0444004d50 0x7f04440051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045a59c700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0444003a90 0x7f0444003e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0450009c00 tx=0x7f04500091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf89307a3179d6a2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045000e710 con 0x7f0444003a90 2023-10-06T04:31:50.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f045000e870 con 0x7f0444003a90 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.191+0000 7f045959a700 1 -- 172.21.15.123:0/4285222739 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f045000ec30 con 0x7f0444003a90 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.192+0000 7f0460dcb700 1 -- 172.21.15.123:0/4285222739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0444003a90 msgr2=0x7f0444003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.192+0000 7f0460dcb700 1 --2- 172.21.15.123:0/4285222739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0444003a90 0x7f0444003e60 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f0450009c00 tx=0x7f04500091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.192+0000 7f045ad9d700 1 -- 172.21.15.123:0/4285222739 reap_dead start 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.192+0000 7f0460dcb700 1 -- 172.21.15.123:0/4285222739 shutdown_connections 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.192+0000 7f0460dcb700 1 -- 172.21.15.123:0/4285222739 >> 172.21.15.123:0/4285222739 conn(0x7f045c0c9c90 msgr2=0x7f045c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.192+0000 7f0460dcb700 1 -- 172.21.15.123:0/4285222739 shutdown_connections 2023-10-06T04:31:50.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.192+0000 7f0460dcb700 1 -- 172.21.15.123:0/4285222739 wait complete. 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.193+0000 7f0460dcb700 1 Processor -- start 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.193+0000 7f0460dcb700 1 -- start start 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0444003a90 0x7f045c150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c0c4e20 0x7f045c0af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f045c0afbe0 0x7f045c14f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0b0120 con 0x7f045c0c4e20 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0b0400 con 0x7f045c0afbe0 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f045c0b06e0 con 0x7f0444003a90 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c0c4e20 0x7f045c0af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c0c4e20 0x7f045c0af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:43740/0 (socket says 172.21.15.123:43740) 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 -- 172.21.15.123:0/718838384 learned_addr learned my addr 172.21.15.123:0/718838384 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 -- 172.21.15.123:0/718838384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0444003a90 msgr2=0x7f045c150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0444003a90 0x7f045c150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 -- 172.21.15.123:0/718838384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f045c0afbe0 msgr2=0x7f045c14f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f045c0afbe0 0x7f045c14f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 -- 172.21.15.123:0/718838384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f045c157100 con 0x7f045c0c4e20 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0459d9b700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c0c4e20 0x7f045c0af6a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f044c002a90 tx=0x7f044c007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f044a7fc700 1 -- 172.21.15.123:0/718838384 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c00f9a0 con 0x7f045c0c4e20 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f044a7fc700 1 -- 172.21.15.123:0/718838384 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044c00fb00 con 0x7f045c0c4e20 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f045c150f60 con 0x7f045c0c4e20 2023-10-06T04:31:50.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f044a7fc700 1 -- 172.21.15.123:0/718838384 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044c00fec0 con 0x7f045c0c4e20 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.194+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f045c0c5290 con 0x7f045c0c4e20 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.195+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f045c1572f0 con 0x7f045c0c4e20 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.195+0000 7f044a7fc700 1 -- 172.21.15.123:0/718838384 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f044c013600 con 0x7f045c0c4e20 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.196+0000 7f044a7fc700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f043c066f30 0x7f043c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.196+0000 7f044a7fc700 1 -- 172.21.15.123:0/718838384 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f044c084d30 con 0x7f045c0c4e20 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.196+0000 7f045a59c700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f043c066f30 0x7f043c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.196+0000 7f045a59c700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f043c066f30 0x7f043c0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f04500091e0 tx=0x7f045001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:50.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.200+0000 7f044a7fc700 1 -- 172.21.15.123:0/718838384 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f044c051e40 con 0x7f045c0c4e20 2023-10-06T04:31:50.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.423+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f045c06b610 con 0x7f045c0c4e20 2023-10-06T04:31:50.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.424+0000 7f044a7fc700 1 -- 172.21.15.123:0/718838384 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f044c017090 con 0x7f045c0c4e20 2023-10-06T04:31:50.426 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:31:50.427 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f043c066f30 msgr2=0x7f043c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f043c066f30 0x7f043c0693e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f04500091e0 tx=0x7f045001a000 comp rx=0 tx=0).stop 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c0c4e20 msgr2=0x7f045c0af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c0c4e20 0x7f045c0af6a0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7f044c002a90 tx=0x7f044c007d10 comp rx=0 tx=0).stop 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 shutdown_connections 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f043c066f30 0x7f043c0693e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f045c0afbe0 0x7f045c14f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0444003a90 0x7f045c150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 --2- 172.21.15.123:0/718838384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f045c0c4e20 0x7f045c0af6a0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 >> 172.21.15.123:0/718838384 conn(0x7f045c0c9c90 msgr2=0x7f045c0c3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:50.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 shutdown_connections 2023-10-06T04:31:50.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.426+0000 7f0460dcb700 1 -- 172.21.15.123:0/718838384 wait complete. 2023-10-06T04:31:50.431 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:31:50.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:31:34.419212+0000', 'modified': '2023-10-06T04:31:38.254397+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4293}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4293': {'gid': 4293, 'name': 'c', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.123:6837/2210806135', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6836', 'nonce': 2210806135}, {'type': 'v1', 'addr': '172.21.15.123:6837', 'nonce': 2210806135}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [4], 'metadata_pool': 3, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:31:50.845 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:31:50.845 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-10-06T04:31:50.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.969+0000 7f3d70c67700 1 Processor -- start 2023-10-06T04:31:50.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.969+0000 7f3d70c67700 1 -- start start 2023-10-06T04:31:50.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d70c67700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3d6c1560e0 0x7f3d6c06b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d70c67700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d6c06e960 0x7f3d6c068c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d70c67700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3d6c06ab80 con 0x7f3d6c068770 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d70c67700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c06f1f0 con 0x7f3d6c06e960 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d70c67700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3d6c069150 con 0x7f3d6c1560e0 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d6ad9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3d6c06e960 0x7f3d6c068c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d69d9b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3d6c068770 0x7f3d6c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58098/0 (socket says 172.21.15.123:58098) 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.970+0000 7f3d69d9b700 1 -- 172.21.15.123:0/4218276096 learned_addr learned my addr 172.21.15.123:0/4218276096 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/4218276096 >> v2:172.21.15.177:3300/0 conn(0x7f3d6c1560e0 0x7f3d6c06b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/4218276096 >> v2:172.21.15.177:3300/0 conn(0x7f3d6c1560e0 msgr2=0x7f3d6c06b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/4218276096 >> v2:172.21.15.177:3300/0 conn(0x7f3d6c1560e0 0x7f3d6c06b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/4218276096 >> v1:172.21.15.123:6789/0 conn(0x7f3d6c068770 legacy=0x7f3d6c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/4218276096 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d6c06ef10 con 0x7f3d6c06e960 2023-10-06T04:31:50.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6959a700 1 -- 172.21.15.123:0/4218276096 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2120694502 0 0) 0x7f3d6c06ab80 con 0x7f3d6c068770 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/4218276096 >> v2:172.21.15.177:3300/0 conn(0x7f3d6c1560e0 0x7f3d6c06b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/4218276096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c068c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06de50 tx=0x7f3d64009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c93560d67a1906e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6959a700 1 -- 172.21.15.123:0/4218276096 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400e070 con 0x7f3d6c06e960 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6959a700 1 -- 172.21.15.123:0/4218276096 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d6400bc30 con 0x7f3d6c06e960 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d6959a700 1 -- 172.21.15.123:0/4218276096 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64012430 con 0x7f3d6c06e960 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d70c67700 1 -- 172.21.15.123:0/4218276096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 msgr2=0x7f3d6c068c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.971+0000 7f3d70c67700 1 --2- 172.21.15.123:0/4218276096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c068c10 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d6c06de50 tx=0x7f3d64009780 comp rx=0 tx=0).stop 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.972+0000 7f3d70c67700 1 -- 172.21.15.123:0/4218276096 shutdown_connections 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.972+0000 7f3d70c67700 1 --2- 172.21.15.123:0/4218276096 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c068c10 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.972+0000 7f3d70c67700 1 --2- 172.21.15.123:0/4218276096 >> v2:172.21.15.177:3300/0 conn(0x7f3d6c1560e0 0x7f3d6c06b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.972+0000 7f3d70c67700 1 -- 172.21.15.123:0/4218276096 >> 172.21.15.123:0/4218276096 conn(0x7f3d6c0cd010 msgr2=0x7f3d6c0cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:50.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.972+0000 7f3d70c67700 1 -- 172.21.15.123:0/4218276096 shutdown_connections 2023-10-06T04:31:50.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.972+0000 7f3d70c67700 1 -- 172.21.15.123:0/4218276096 wait complete. 2023-10-06T04:31:50.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.973+0000 7f3d70c67700 1 Processor -- start 2023-10-06T04:31:50.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.973+0000 7f3d70c67700 1 -- start start 2023-10-06T04:31:50.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d70c67700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c068770 0x7f3d6c157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d70c67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c0afe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d70c67700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c1560e0 0x7f3d6c0b1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d70c67700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c1564b0 con 0x7f3d6c06e960 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d70c67700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c156790 con 0x7f3d6c1560e0 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d70c67700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3d6c0b0770 con 0x7f3d6c068770 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c0afe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c0afe00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33294/0 (socket says 172.21.15.123:33294) 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 -- 172.21.15.123:0/3442986715 learned_addr learned my addr 172.21.15.123:0/3442986715 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c1560e0 0x7f3d6c0b1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c068770 0x7f3d6c157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 -- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c068770 msgr2=0x7f3d6c157cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c068770 0x7f3d6c157cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 -- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c1560e0 msgr2=0x7f3d6c0b1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c1560e0 0x7f3d6c0b1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.974+0000 7f3d69d9b700 1 -- 172.21.15.123:0/3442986715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3d64009a20 con 0x7f3d6c06e960 2023-10-06T04:31:50.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.975+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c0afe00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c7e0 tx=0x7f3d60007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.975+0000 7f3d5a7fc700 1 -- 172.21.15.123:0/3442986715 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60005b30 con 0x7f3d6c06e960 2023-10-06T04:31:50.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.975+0000 7f3d5a7fc700 1 -- 172.21.15.123:0/3442986715 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d60005c90 con 0x7f3d6c06e960 2023-10-06T04:31:50.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.975+0000 7f3d5a7fc700 1 -- 172.21.15.123:0/3442986715 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d60007300 con 0x7f3d6c06e960 2023-10-06T04:31:50.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.975+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3d6c0b0a50 con 0x7f3d6c06e960 2023-10-06T04:31:50.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.975+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c158230 con 0x7f3d6c06e960 2023-10-06T04:31:50.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.976+0000 7f3d5a7fc700 1 -- 172.21.15.123:0/3442986715 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3d6001b400 con 0x7f3d6c06e960 2023-10-06T04:31:50.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.976+0000 7f3d5a7fc700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3d4c066f30 0x7f3d4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.977+0000 7f3d5a7fc700 1 -- 172.21.15.123:0/3442986715 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d600841a0 con 0x7f3d6c06e960 2023-10-06T04:31:50.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.977+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3d4c066f30 0x7f3d4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.977+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3d4c066f30 0x7f3d4c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:50.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 Processor -- start 2023-10-06T04:31:50.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 -- start start 2023-10-06T04:31:50.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc023fb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02409f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc023a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc0158310 con 0x559fc023a260 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc0195130 con 0x559fc0289370 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fc0192af0 con 0x559fc023f770 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.979+0000 7f3d6ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc023a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc023a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33302/0 (socket says 172.21.15.123:33302) 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/352741224 learned_addr learned my addr 172.21.15.123:0/352741224 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02409f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc023fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 msgr2=0x559fc023fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc023fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 msgr2=0x559fc02409f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02409f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/352741224 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fbff94790 con 0x559fc023a260 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc023a6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d640097c0 tx=0x7f3d64019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a0bc3262c449001 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d4b7fe700 1 -- 172.21.15.123:0/352741224 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d6400be70 con 0x559fc023a260 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d4b7fe700 1 -- 172.21.15.123:0/352741224 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d64012810 con 0x559fc023a260 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.980+0000 7f3d4b7fe700 1 -- 172.21.15.123:0/352741224 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64012bd0 con 0x559fc023a260 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/352741224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 msgr2=0x559fc023a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc023a6d0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f3d640097c0 tx=0x7f3d64019000 comp rx=0 tx=0).stop 2023-10-06T04:31:50.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/352741224 shutdown_connections 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc023a6d0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc023fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/352741224 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02409f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/352741224 >> 172.21.15.123:0/352741224 conn(0x559fc0288890 msgr2=0x559fc0288c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/352741224 shutdown_connections 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/352741224 wait complete. 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.981+0000 7f3d79f7bb80 1 -- start start 2023-10-06T04:31:50.992 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.982+0000 7f3d79f7bb80 1 Processor -- start 2023-10-06T04:31:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.982+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc0262240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.982+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0265de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.982+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc0289370 0x559fc0266320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.982+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc018e8e0 con 0x559fc023a260 2023-10-06T04:31:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.982+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc013a980 con 0x559fc023f770 2023-10-06T04:31:50.996 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.982+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fc013a620 con 0x559fc0289370 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d79f7bb80 10 client.0 _ll_register_callbacks cb 0x559fbff9da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc0262240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc0262240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33308/0 (socket says 172.21.15.123:33308) 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 -- 172.21.15.123:0/1504913053 learned_addr learned my addr 172.21.15.123:0/1504913053 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0265de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc0289370 0x559fc0266320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 -- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc0289370 msgr2=0x559fc0266320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc0289370 0x559fc0266320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 -- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 msgr2=0x559fc0265de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:50.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0265de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 -- 172.21.15.123:0/1504913053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3d6c06ef10 con 0x559fc023a260 2023-10-06T04:31:51.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0265de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:51.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc0262240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0147e0 tx=0x7f3d5c018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e548ababf6de610b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d48ff9700 1 -- 172.21.15.123:0/1504913053 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c021630 con 0x559fc023a260 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d48ff9700 1 -- 172.21.15.123:0/1504913053 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d5c021790 con 0x559fc023a260 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fc0232e50 con 0x559fc023a260 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d48ff9700 1 -- 172.21.15.123:0/1504913053 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d5c021b60 con 0x559fc023a260 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d79f7bb80 10 client.4704 fetch_fsmap learned FSMap version 8 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x559fc00810a0 con 0x559fc023a260 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d48ff9700 1 -- 172.21.15.123:0/1504913053 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559fc0232e50 con 0x559fc023a260 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.983+0000 7f3d48ff9700 1 -- 172.21.15.123:0/1504913053 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3d5c021cc0 con 0x559fc023a260 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d79f7bb80 10 client.4704 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T04:31:51.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d79f7bb80 10 client.4704 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d79f7bb80 20 client.4704 populate_metadata read hostname 'smithi123' 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc023a860 0x559fc023ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d79f7bb80 4 client.4704 mds_command: new command op to 4293 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 --> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559fc020fc20 con 0x559fc023a860 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc023a860 0x559fc023ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d48ff9700 10 client.4704 ms_handle_connect on v2:172.21.15.123:6836/2210806135 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.984+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc023a860 0x559fc023ac30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de4d23c0bb9b8d4e server_cookie=60ff0b118c64d34c in_seq=0 out_seq=0 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d48ff9700 1 -- 172.21.15.123:0/1504913053 <== mds.0 v2:172.21.15.123:6836/2210806135 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x559fc020fc20 con 0x559fc023a860 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d48ff9700 10 client.4704 handle_command_reply: tid=0 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 client.4704 shutdown 2023-10-06T04:31:51.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 msgr2=0x559fc0262240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc0262240 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c0147e0 tx=0x7f3d5c018040 comp rx=0 tx=0).stop 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 shutdown_connections 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc023a860 0x559fc023ac30 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc0262240 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc0289370 0x559fc0266320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d48ff9700 0 client.4704 ms_handle_reset on v2:172.21.15.123:6836/2210806135 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/1504913053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0265de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 >> 172.21.15.123:0/1504913053 conn(0x559fc0288890 msgr2=0x559fc0285f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 shutdown_connections 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/1504913053 wait complete. 2023-10-06T04:31:51.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:50.985+0000 7f3d79f7bb80 20 client.4704 trim_cache size 0 max 16384 2023-10-06T04:31:51.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 Processor -- start 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 -- start start 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc029e260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc0266be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc00261e0 con 0x559fc023a260 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc0263cd0 con 0x559fc0289370 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.002+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fc0240cf0 con 0x559fc023f770 2023-10-06T04:31:51.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc029e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc029e260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33312/0 (socket says 172.21.15.123:33312) 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 -- 172.21.15.123:0/2047617676 learned_addr learned my addr 172.21.15.123:0/2047617676 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 -- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 msgr2=0x559fc0266be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc0266be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc0266be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 -- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 msgr2=0x559fc02611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 -- 172.21.15.123:0/2047617676 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559fc015d3c0 con 0x559fc023a260 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:51.016 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc029e260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00f5d0 tx=0x7f3d5c00ffd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5587684103a84f3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d4bfff700 1 -- 172.21.15.123:0/2047617676 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c021630 con 0x559fc023a260 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d4bfff700 1 -- 172.21.15.123:0/2047617676 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3d5c021790 con 0x559fc023a260 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d4bfff700 1 -- 172.21.15.123:0/2047617676 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d5c021d70 con 0x559fc023a260 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/2047617676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 msgr2=0x559fc029e260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.003+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc029e260 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c00f5d0 tx=0x7f3d5c00ffd0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/2047617676 shutdown_connections 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc023a260 0x559fc029e260 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc0289370 0x559fc02611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/2047617676 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023f770 0x559fc0266be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/2047617676 >> 172.21.15.123:0/2047617676 conn(0x559fc0288890 msgr2=0x559fc023bfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:51.017 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/2047617676 shutdown_connections 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/2047617676 wait complete. 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 -- start start 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.004+0000 7f3d79f7bb80 1 Processor -- start 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023a260 0x559fc02611d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0261710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d79f7bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc0289370 0x559fc0265f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc026af40 con 0x559fc0289370 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559fc02eb300 con 0x559fc023f770 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d79f7bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559fc02ea870 con 0x559fc023a260 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d79f7bb80 10 client.0 _ll_register_callbacks cb 0x559fbff9da00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:31:51.018 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc0289370 0x559fc0265f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc0289370 0x559fc0265f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33322/0 (socket says 172.21.15.123:33322) 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/633538841 learned_addr learned my addr 172.21.15.123:0/633538841 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d69d9b700 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0261710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023a260 0x559fc02611d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023a260 msgr2=0x559fc02611d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023a260 0x559fc02611d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 msgr2=0x559fc0261710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.005+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0261710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d6ad9d700 1 -- 172.21.15.123:0/633538841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559fc00810a0 con 0x559fc0289370 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d6ad9d700 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc0289370 0x559fc0265f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d64000f80 tx=0x7f3d64020040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17c0372039c5eae9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d48ff9700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:31:51.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d48ff9700 1 -- 172.21.15.123:0/633538841 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3d64030430 con 0x559fc0289370 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d48ff9700 1 -- 172.21.15.123:0/633538841 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3d64030590 con 0x559fc0289370 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559fc023cec0 con 0x559fc0289370 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d48ff9700 1 -- 172.21.15.123:0/633538841 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7f3d64030960 con 0x559fc0289370 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 10 client.4710 fetch_fsmap learned FSMap version 8 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x559fbfff51d0 con 0x559fc0289370 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d48ff9700 1 -- 172.21.15.123:0/633538841 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559fc023cec0 con 0x559fc0289370 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d48ff9700 1 -- 172.21.15.123:0/633538841 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 8) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f3d6400b3e0 con 0x559fc0289370 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 10 client.4710 fetch_fsmap finished waiting for FSMap version 8 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 10 client.4710 resolve_mds: resolved name 'c' to daemon mds.c 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 20 client.4710 populate_metadata read hostname 'smithi123' 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc0266470 0x559fc0269b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 4 client.4710 mds_command: new command op to 4293 tid=0[{"prefix": "status"}] 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 --> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] -- command(tid 0: {"prefix": "status"}) v1 -- 0x7f3d64012810 con 0x559fc0266470 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.006+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc0266470 0x559fc0269b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d48ff9700 10 client.4710 ms_handle_connect on v2:172.21.15.123:6836/2210806135 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d6a59c700 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc0266470 0x559fc0269b20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d709eec03e310d8 server_cookie=c8384bda2f19d8e1 in_seq=0 out_seq=0 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d48ff9700 1 -- 172.21.15.123:0/633538841 <== mds.0 v2:172.21.15.123:6836/2210806135 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x7f3d64012810 con 0x559fc0266470 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d48ff9700 10 client.4710 handle_command_reply: tid=0 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 client.4710 shutdown 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc0289370 msgr2=0x559fc0265f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc0289370 0x559fc0265f30 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f3d64000f80 tx=0x7f3d64020040 comp rx=0 tx=0).stop 2023-10-06T04:31:51.021 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 shutdown_connections 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:6836/2210806135,v1:172.21.15.123:6837/2210806135] conn(0x559fc0266470 0x559fc0269b20 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559fc0289370 0x559fc0265f30 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d48ff9700 0 client.4710 ms_handle_reset on v2:172.21.15.123:6836/2210806135 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559fc023a260 0x559fc02611d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 --2- 172.21.15.123:0/633538841 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559fc023f770 0x559fc0261710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 >> 172.21.15.123:0/633538841 conn(0x559fc0288890 msgr2=0x559fc029e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 shutdown_connections 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 1 -- 172.21.15.123:0/633538841 wait complete. 2023-10-06T04:31:51.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d79f7bb80 20 client.4710 trim_cache size 0 max 16384 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "1cea82d3-d2ef-4e20-8c3e-67296f7df92c", 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4293, 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 15.765268125, 2023-10-06T04:31:51.023 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 8, 2023-10-06T04:31:51.024 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 23, 2023-10-06T04:31:51.024 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 23, 2023-10-06T04:31:51.024 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 21.885803889000002 2023-10-06T04:31:51.024 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:31:51.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3d4c066f30 msgr2=0x7f3d4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d70c67700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3d4c066f30 0x7f3d4c0693e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f3d5c006fd0 tx=0x7f3d5c008040 comp rx=0 tx=0).stop 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 msgr2=0x7f3d6c0afe00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.007+0000 7f3d70c67700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c0afe00 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f3d6000c7e0 tx=0x7f3d60007df0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 shutdown_connections 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3d4c066f30 0x7f3d4c0693e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3d6c1560e0 0x7f3d6c0b1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3d6c068770 0x7f3d6c157cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 --2- 172.21.15.123:0/3442986715 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3d6c06e960 0x7f3d6c0afe00 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 >> 172.21.15.123:0/3442986715 conn(0x7f3d6c0cd010 msgr2=0x7f3d6c0cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:51.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 shutdown_connections 2023-10-06T04:31:51.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.008+0000 7f3d70c67700 1 -- 172.21.15.123:0/3442986715 wait complete. 2023-10-06T04:31:51.027 INFO:teuthology.run_tasks:Running task ceph-fuse... 2023-10-06T04:31:51.050 INFO:tasks.ceph_fuse:Running ceph_fuse task... 2023-10-06T04:31:51.050 INFO:tasks.ceph_fuse:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-10-06T04:31:51.053 INFO:tasks.ceph_fuse:client.0 config is {} 2023-10-06T04:31:51.053 INFO:tasks.ceph_fuse:client.2 config is {} 2023-10-06T04:31:51.053 INFO:tasks.ceph_fuse:client.1 config is {} 2023-10-06T04:31:51.053 INFO:tasks.ceph_fuse:client.3 config is {} 2023-10-06T04:31:51.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.054 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-10-06T04:31:51.074 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.074 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-10-06T04:31:51.143 INFO:teuthology.orchestra.run.smithi123.stderr:Cannot find device "ceph-brx" 2023-10-06T04:31:51.144 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:31:51.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.145 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T04:31:51.169 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.169 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-10-06T04:31:51.244 INFO:teuthology.orchestra.run.smithi177.stderr:Cannot find device "ceph-brx" 2023-10-06T04:31:51.245 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:31:51.245 INFO:tasks.ceph_fuse:Mounting ceph-fuse clients... 2023-10-06T04:31:51.245 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:31:51.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.362+0000 7f72858ee700 1 Processor -- start 2023-10-06T04:31:51.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.362+0000 7f72858ee700 1 -- start start 2023-10-06T04:31:51.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.362+0000 7f72858ee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72800adf70 0x7f72800c6ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.362+0000 7f72858ee700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800abc60 con 0x7f72800698e0 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.362+0000 7f72858ee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72800c81b0 con 0x7f72800adba0 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f72858ee700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72800c75a0 con 0x7f72800adf70 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727e7fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72800adf70 0x7f72800c6ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727f7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f72800698e0 0x7f72800abed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58112/0 (socket says 172.21.15.123:58112) 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727f7fe700 1 -- 172.21.15.123:0/112414116 learned_addr learned my addr 172.21.15.123:0/112414116 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727e7fc700 1 -- 172.21.15.123:0/112414116 >> v1:172.21.15.177:6789/0 conn(0x7f72800adba0 legacy=0x7f72800691d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727e7fc700 1 -- 172.21.15.123:0/112414116 >> v1:172.21.15.123:6789/0 conn(0x7f72800698e0 legacy=0x7f72800abed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727dffb700 1 -- 172.21.15.123:0/112414116 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 824590071 0 0) 0x7f72800abc60 con 0x7f72800698e0 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727e7fc700 1 -- 172.21.15.123:0/112414116 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72800c7880 con 0x7f72800adf70 2023-10-06T04:31:51.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727e7fc700 1 --2- 172.21.15.123:0/112414116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72800c6ec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f72740098e0 tx=0x7f72740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f0fd85d164ed23b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:51.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727dffb700 1 -- 172.21.15.123:0/112414116 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7274009e80 con 0x7f72800adf70 2023-10-06T04:31:51.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727dffb700 1 -- 172.21.15.123:0/112414116 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727400b860 con 0x7f72800adf70 2023-10-06T04:31:51.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.363+0000 7f727dffb700 1 -- 172.21.15.123:0/112414116 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727400bbe0 con 0x7f72800adf70 2023-10-06T04:31:51.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.364+0000 7f72858ee700 1 -- 172.21.15.123:0/112414116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 msgr2=0x7f72800c6ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.364+0000 7f72858ee700 1 --2- 172.21.15.123:0/112414116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72800c6ec0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f72740098e0 tx=0x7f72740091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.364+0000 7f72858ee700 1 -- 172.21.15.123:0/112414116 shutdown_connections 2023-10-06T04:31:51.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.364+0000 7f72858ee700 1 --2- 172.21.15.123:0/112414116 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72800c6ec0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.364+0000 7f72858ee700 1 -- 172.21.15.123:0/112414116 >> 172.21.15.123:0/112414116 conn(0x7f72800680f0 msgr2=0x7f72800cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:51.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.364+0000 7f72858ee700 1 -- 172.21.15.123:0/112414116 shutdown_connections 2023-10-06T04:31:51.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.364+0000 7f72858ee700 1 -- 172.21.15.123:0/112414116 wait complete. 2023-10-06T04:31:51.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.365+0000 7f72858ee700 1 Processor -- start 2023-10-06T04:31:51.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.365+0000 7f72858ee700 1 -- start start 2023-10-06T04:31:51.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72800698e0 0x7f72801493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800adba0 0x7f72801584d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72801421a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800ab900 con 0x7f72800adf70 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72800af160 con 0x7f72800adba0 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72800af440 con 0x7f72800698e0 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72801421a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72801421a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33332/0 (socket says 172.21.15.123:33332) 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 -- 172.21.15.123:0/1730480787 learned_addr learned my addr 172.21.15.123:0/1730480787 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727e7fc700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800adba0 0x7f72801584d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727effd700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72800698e0 0x7f72801493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 -- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72800698e0 msgr2=0x7f72801493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72800698e0 0x7f72801493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 -- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800adba0 msgr2=0x7f72801584d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800adba0 0x7f72801584d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 -- 172.21.15.123:0/1730480787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7274009420 con 0x7f72800adf70 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727effd700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72800698e0 0x7f72801493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f727f7fe700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72801421a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f727800cc90 tx=0x7f7278001a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f7266ffd700 1 -- 172.21.15.123:0/1730480787 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800a960 con 0x7f72800adf70 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f7266ffd700 1 -- 172.21.15.123:0/1730480787 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f727800aac0 con 0x7f72800adf70 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72800af720 con 0x7f72800adf70 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f7266ffd700 1 -- 172.21.15.123:0/1730480787 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f727800ae80 con 0x7f72800adf70 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.366+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72800afce0 con 0x7f72800adf70 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.367+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72800acfe0 con 0x7f72800adf70 2023-10-06T04:31:51.374 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.368+0000 7f7266ffd700 1 -- 172.21.15.123:0/1730480787 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7278007300 con 0x7f72800adf70 2023-10-06T04:31:51.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.368+0000 7f7266ffd700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f726c066f30 0x7f726c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:51.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.368+0000 7f727effd700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f726c066f30 0x7f726c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:51.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.368+0000 7f7266ffd700 1 -- 172.21.15.123:0/1730480787 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7278084a70 con 0x7f72800adf70 2023-10-06T04:31:51.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.369+0000 7f727effd700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f726c066f30 0x7f726c0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7270007fd0 tx=0x7f7270009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:51.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.372+0000 7f7266ffd700 1 -- 172.21.15.123:0/1730480787 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7278051c00 con 0x7f72800adf70 2023-10-06T04:31:51.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.596+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f72800afed0 con 0x7f72800adf70 2023-10-06T04:31:51.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.597+0000 7f7266ffd700 1 -- 172.21.15.123:0/1730480787 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7278055220 con 0x7f72800adf70 2023-10-06T04:31:51.599 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:31:51.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.598+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f726c066f30 msgr2=0x7f726c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.598+0000 7f72858ee700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f726c066f30 0x7f726c0693e0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f7270007fd0 tx=0x7f7270009040 comp rx=0 tx=0).stop 2023-10-06T04:31:51.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.598+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 msgr2=0x7f72801421a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:51.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.598+0000 7f72858ee700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72801421a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f727800cc90 tx=0x7f7278001a80 comp rx=0 tx=0).stop 2023-10-06T04:31:51.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 shutdown_connections 2023-10-06T04:31:51.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f726c066f30 0x7f726c0693e0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72800adf70 0x7f72801421a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72800698e0 0x7f72801493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 --2- 172.21.15.123:0/1730480787 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f72800adba0 0x7f72801584d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:51.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 >> 172.21.15.123:0/1730480787 conn(0x7f72800680f0 msgr2=0x7f72800ac890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:51.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 shutdown_connections 2023-10-06T04:31:51.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:51.599+0000 7f72858ee700 1 -- 172.21.15.123:0/1730480787 wait complete. 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:31:51.613 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:31:51.614 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:31:51.614 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.614 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:51.632 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42503sec preferred_lft 42503sec 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2023-10-06T04:31:51.633 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:51.634 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T04:31:51.634 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.634 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:51.634 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:51.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add name ceph-brx type bridge 2023-10-06T04:31:51.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr flush dev ceph-brx 2023-10-06T04:31:51.634 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set ceph-brx up 2023-10-06T04:31:51.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T04:31:51.635 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:51.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.794 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:31:51.825 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-10-06T04:31:51.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.827 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-10-06T04:31:51.912 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-10-06T04:31:51.912 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:31:51.912 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:31:51.913 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:31:51.913 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:31:51.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:51.913 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:51.913 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:51.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:31:51.913 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:31:51.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:31:51.914 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:52.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:52.179 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-10-06T04:31:52.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:52.198 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-10-06T04:31:52.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:52.257 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:52.257 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:52.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:31:52.257 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-10-06T04:31:52.257 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:52.352 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-10-06T04:31:52.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-10-06T04:31:52.353 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:52.547 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:52.548 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:52.548 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:52.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2023-10-06T04:31:52.548 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T04:31:52.548 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:52.645 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:31:52.646 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:52.646 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:31:52.664 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:31:52.665 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:52.665 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:31:52.724 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:31:52.792 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:52.792 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:52.858 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:52.859 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:52.860 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:52.860 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:31:52.919 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:31:52.919 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:31:52.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:31:52.987 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:52.988 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:53.045 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:2023-10-06T04:31:53.043+0000 7feb1c74f540 -1 init, newargv = 0x5586e650fab0 newargc=15 2023-10-06T04:31:53.046 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[129740]: starting ceph client 2023-10-06T04:31:53.050 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[129740]: starting fuse 2023-10-06T04:31:53.058 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:53.059 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:53.060 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:53.060 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:31:53.118 INFO:teuthology.orchestra.run.smithi123.stdout:46 2023-10-06T04:31:53.118 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> import re 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:53.119 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2023-10-06T04:31:53.120 DEBUG:teuthology.orchestra.run.smithi123:> return f 2023-10-06T04:31:53.121 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:31:53.121 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:53.121 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.0")) 2023-10-06T04:31:53.121 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-10-06T04:31:53.241 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.0.129740.asok 2023-10-06T04:31:53.423 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.129740.asok 2023-10-06T04:31:53.424 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:53.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.129740.asok status 2023-10-06T04:31:53.576 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "0", 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "129740", 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2023-10-06T04:31:53.577 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4725, 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4725 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2604721088 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:31:53.578 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 2604721088 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4725 192.168.0.1:0/2604721088", 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/2604721088", 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 8, 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 23, 2023-10-06T04:31:53.579 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:31:53.580 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2023-10-06T04:31:53.580 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2023-10-06T04:31:53.580 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:31:53.586 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:31:53.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.700+0000 7fafe19dd700 1 Processor -- start 2023-10-06T04:31:53.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.700+0000 7fafe19dd700 1 -- start start 2023-10-06T04:31:53.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafe19dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc1591b0 0x7fafdc159580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:53.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafe19dd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafdc0b43b0 0x7fafdc0cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:53.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafe19dd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fafdc15b370 con 0x7fafdc0b05b0 2023-10-06T04:31:53.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafe19dd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fafdc15b0d0 con 0x7fafdc0b43b0 2023-10-06T04:31:53.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafe19dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0b0fe0 con 0x7fafdc1591b0 2023-10-06T04:31:53.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdb7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafdc0b43b0 0x7fafdc0cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:53.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdb7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fafdc0b43b0 0x7fafdc0cf380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33348/0 (socket says 172.21.15.123:33348) 2023-10-06T04:31:53.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdb7fe700 1 -- 172.21.15.123:0/563068124 learned_addr learned my addr 172.21.15.123:0/563068124 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:53.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdaffd700 1 --2- 172.21.15.123:0/563068124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc1591b0 0x7fafdc159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:53.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdb7fe700 1 -- 172.21.15.123:0/563068124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc1591b0 msgr2=0x7fafdc159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:53.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdb7fe700 1 --2- 172.21.15.123:0/563068124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc1591b0 0x7fafdc159580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdb7fe700 1 -- 172.21.15.123:0/563068124 >> v1:172.21.15.177:6789/0 conn(0x7fafdc0b05b0 legacy=0x7fafdc0b3ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:53.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.701+0000 7fafdb7fe700 1 -- 172.21.15.123:0/563068124 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafdc0b12c0 con 0x7fafdc0b43b0 2023-10-06T04:31:53.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafdb7fe700 1 --2- 172.21.15.123:0/563068124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc0cf380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009f20 tx=0x7fafd4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1cb21bf9700278a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:53.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafd9ffb700 1 -- 172.21.15.123:0/563068124 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd400e070 con 0x7fafdc0b43b0 2023-10-06T04:31:53.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafd9ffb700 1 -- 172.21.15.123:0/563068124 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafd400ba40 con 0x7fafdc0b43b0 2023-10-06T04:31:53.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafd9ffb700 1 -- 172.21.15.123:0/563068124 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafd40092e0 con 0x7fafdc0b43b0 2023-10-06T04:31:53.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 -- 172.21.15.123:0/563068124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 msgr2=0x7fafdc0cf380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:53.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 --2- 172.21.15.123:0/563068124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc0cf380 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fafd4009f20 tx=0x7fafd4009840 comp rx=0 tx=0).stop 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 -- 172.21.15.123:0/563068124 shutdown_connections 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 --2- 172.21.15.123:0/563068124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc0cf380 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 --2- 172.21.15.123:0/563068124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc1591b0 0x7fafdc159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 -- 172.21.15.123:0/563068124 >> 172.21.15.123:0/563068124 conn(0x7fafdc0995a0 msgr2=0x7fafdc063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 -- 172.21.15.123:0/563068124 shutdown_connections 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.702+0000 7fafe19dd700 1 -- 172.21.15.123:0/563068124 wait complete. 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 Processor -- start 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 -- start start 2023-10-06T04:31:53.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc0b05b0 0x7fafdc149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc16a170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafdc1591b0 0x7fafdc156180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc0abf00 con 0x7fafdc0b43b0 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc09b120 con 0x7fafdc1591b0 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafe19dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafdc16a6b0 con 0x7fafdc0b05b0 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc16a170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafda7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc16a170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33360/0 (socket says 172.21.15.123:33360) 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.704+0000 7fafda7fc700 1 -- 172.21.15.123:0/3643677823 learned_addr learned my addr 172.21.15.123:0/3643677823 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafdb7fe700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafdc1591b0 0x7fafdc156180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafdaffd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc0b05b0 0x7fafdc149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:53.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafda7fc700 1 -- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc0b05b0 msgr2=0x7fafdc149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafda7fc700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc0b05b0 0x7fafdc149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafda7fc700 1 -- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafdc1591b0 msgr2=0x7fafdc156180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafda7fc700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafdc1591b0 0x7fafdc156180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafda7fc700 1 -- 172.21.15.123:0/3643677823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafd4009ae0 con 0x7fafdc0b43b0 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafdaffd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc0b05b0 0x7fafdc149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafda7fc700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc16a170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00b5e0 tx=0x7fafcc009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafc2ffd700 1 -- 172.21.15.123:0/3643677823 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00f510 con 0x7fafdc0b43b0 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafc2ffd700 1 -- 172.21.15.123:0/3643677823 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fafcc00f670 con 0x7fafdc0b43b0 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fafdc09a510 con 0x7fafdc0b43b0 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafc2ffd700 1 -- 172.21.15.123:0/3643677823 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fafcc00fa30 con 0x7fafdc0b43b0 2023-10-06T04:31:53.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.705+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fafdc09aad0 con 0x7fafdc0b43b0 2023-10-06T04:31:53.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.706+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fafdc0b15a0 con 0x7fafdc0b43b0 2023-10-06T04:31:53.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.706+0000 7fafc2ffd700 1 -- 172.21.15.123:0/3643677823 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fafcc01a400 con 0x7fafdc0b43b0 2023-10-06T04:31:53.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.707+0000 7fafc2ffd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fafc8066f30 0x7fafc80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:53.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.707+0000 7fafdaffd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fafc8066f30 0x7fafc80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:53.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.707+0000 7fafc2ffd700 1 -- 172.21.15.123:0/3643677823 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fafcc084260 con 0x7fafdc0b43b0 2023-10-06T04:31:53.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.707+0000 7fafdaffd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fafc8066f30 0x7fafc80693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafd0008a80 tx=0x7fafd0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:53.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.711+0000 7fafc2ffd700 1 -- 172.21.15.123:0/3643677823 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fafcc0513f0 con 0x7fafdc0b43b0 2023-10-06T04:31:53.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.934+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fafdc04bb80 con 0x7fafdc0b43b0 2023-10-06T04:31:53.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.935+0000 7fafc2ffd700 1 -- 172.21.15.123:0/3643677823 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fafcc054a10 con 0x7fafdc0b43b0 2023-10-06T04:31:53.938 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:31:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fafc8066f30 msgr2=0x7fafc80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fafc8066f30 0x7fafc80693e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fafd0008a80 tx=0x7fafd0008040 comp rx=0 tx=0).stop 2023-10-06T04:31:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 msgr2=0x7fafdc16a170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc16a170 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7fafcc00b5e0 tx=0x7fafcc009120 comp rx=0 tx=0).stop 2023-10-06T04:31:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 shutdown_connections 2023-10-06T04:31:53.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fafc8066f30 0x7fafc80693e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fafdc1591b0 0x7fafdc156180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fafdc0b05b0 0x7fafdc149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 --2- 172.21.15.123:0/3643677823 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fafdc0b43b0 0x7fafdc16a170 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 >> 172.21.15.123:0/3643677823 conn(0x7fafdc0995a0 msgr2=0x7fafdc0afbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 shutdown_connections 2023-10-06T04:31:53.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:53.936+0000 7fafe19dd700 1 -- 172.21.15.123:0/3643677823 wait complete. 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:31:53.951 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-10-06T04:31:53.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:53.952 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-10-06T04:31:53.971 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:53.971 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:53.971 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:31:53.972 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42501sec preferred_lft 42501sec 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:e0:39:ac:6a:27 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5073:dcff:fe45:d49e/64 scope link 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:e0:39:ac:6a:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:31:53.973 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::44e0:39ff:feac:6a27/64 scope link tentative 2023-10-06T04:31:53.974 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:53.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:53.974 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:31:54.037 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-10-06T04:31:54.038 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.038 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-10-06T04:31:54.098 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-10-06T04:31:54.099 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:31:54.099 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:31:54.099 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:31:54.099 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:31:54.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.100 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:54.100 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:54.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:31:54.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:31:54.100 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:31:54.100 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:54.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.207 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-10-06T04:31:54.226 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:31:54.226 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.227 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list-id 2023-10-06T04:31:54.284 INFO:teuthology.orchestra.run.smithi123.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-10-06T04:31:54.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.285 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:54.285 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:54.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:31:54.285 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-10-06T04:31:54.285 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:54.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T04:31:54.415 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.415 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.416 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:31:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 4a:ee:c9:ec:39:aa brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:31:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:31:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::48ee:c9ff:feec:39aa/64 scope link tentative 2023-10-06T04:31:54.416 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-10-06T04:31:54.506 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:54.506 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 4a:ee:c9:ec:39:aa brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::48ee:c9ff:feec:39aa/64 scope link tentative 2023-10-06T04:31:54.507 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:54.517 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-10-06T04:31:54.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-10-06T04:31:54.518 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:54.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:54.752 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:31:54.752 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:31:54.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.1 up 2023-10-06T04:31:54.752 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T04:31:54.752 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:31:54.849 INFO:tasks.cephfs.fuse_mount:Client client.2 config is {} 2023-10-06T04:31:54.850 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:54.850 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-10-06T04:31:54.868 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-10-06T04:31:54.868 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:54.868 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-10-06T04:31:54.926 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:31:54.993 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:54.994 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:55.061 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:55.061 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:55.062 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:55.062 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:31:55.120 INFO:teuthology.orchestra.run.smithi123.stdout:46 2023-10-06T04:31:55.120 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-10-06T04:31:55.121 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.2 --id 2) 2023-10-06T04:31:55.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:31:55.191 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:55.191 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:55.244 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi123.stderr:2023-10-06T04:31:55.242+0000 7f73c96c7540 -1 init, newargv = 0x56378d4d9e50 newargc=15 2023-10-06T04:31:55.244 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi123.stderr:ceph-fuse[130464]: starting ceph client 2023-10-06T04:31:55.249 INFO:tasks.cephfs.fuse_mount.ceph-fuse.2.smithi123.stderr:ceph-fuse[130464]: starting fuse 2023-10-06T04:31:55.258 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:55.259 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:55.259 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:55.260 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:31:55.318 INFO:teuthology.orchestra.run.smithi123.stdout:46 2023-10-06T04:31:55.318 INFO:teuthology.orchestra.run.smithi123.stdout:48 2023-10-06T04:31:55.318 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> import re 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.2.*.asok" 2023-10-06T04:31:55.319 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.2" 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:31:55.320 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2023-10-06T04:31:55.321 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2023-10-06T04:31:55.321 DEBUG:teuthology.orchestra.run.smithi123:> return f 2023-10-06T04:31:55.321 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:31:55.321 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:31:55.321 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.2")) 2023-10-06T04:31:55.321 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-10-06T04:31:55.441 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.2.130464.asok 2023-10-06T04:31:55.624 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.2.130464.asok 2023-10-06T04:31:55.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:55.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.2.130464.asok status 2023-10-06T04:31:55.777 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:31:55.777 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2023-10-06T04:31:55.777 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T04:31:55.777 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "2", 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.2", 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "130464", 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 4737, 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2023-10-06T04:31:55.778 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 4737 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 506013553 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:31:55.779 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 506013553 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.4737 192.168.0.2:0/506013553", 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.2:0/506013553", 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 8, 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 23, 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2023-10-06T04:31:55.780 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:31:55.787 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:31:55.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.903+0000 7f1c61779700 1 Processor -- start 2023-10-06T04:31:55.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.903+0000 7f1c61779700 1 -- start start 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c61779700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1c5c0688c0 0x7f1c5c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c61779700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c5c158f70 0x7f1c5c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c61779700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c5c069a30 con 0x7f1c5c068c90 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c61779700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c06a890 con 0x7f1c5c158f70 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c61779700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c5c159930 con 0x7f1c5c0688c0 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c5b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c5c158f70 0x7f1c5c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c5b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1c5c158f70 0x7f1c5c1593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33374/0 (socket says 172.21.15.123:33374) 2023-10-06T04:31:55.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.904+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/850016093 learned_addr learned my addr 172.21.15.123:0/850016093 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c5affd700 1 --2- 172.21.15.123:0/850016093 >> v2:172.21.15.177:3300/0 conn(0x7f1c5c0688c0 0x7f1c5c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/850016093 >> v2:172.21.15.177:3300/0 conn(0x7f1c5c0688c0 msgr2=0x7f1c5c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c5b7fe700 1 --2- 172.21.15.123:0/850016093 >> v2:172.21.15.177:3300/0 conn(0x7f1c5c0688c0 0x7f1c5c0c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/850016093 >> v1:172.21.15.177:6789/0 conn(0x7f1c5c068c90 legacy=0x7f1c5c06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c59ffb700 1 -- 172.21.15.123:0/850016093 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285129529 0 0) 0x7f1c5c069a30 con 0x7f1c5c068c90 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/850016093 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c5c069c80 con 0x7f1c5c158f70 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c5b7fe700 1 --2- 172.21.15.123:0/850016093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c1593f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c06f2f0 tx=0x7f1c54009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997f8e7e24cc5510 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c59ffb700 1 -- 172.21.15.123:0/850016093 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5400e070 con 0x7f1c5c158f70 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c59ffb700 1 -- 172.21.15.123:0/850016093 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c5400ba20 con 0x7f1c5c158f70 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.905+0000 7f1c59ffb700 1 -- 172.21.15.123:0/850016093 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c540092e0 con 0x7f1c5c158f70 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 -- 172.21.15.123:0/850016093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 msgr2=0x7f1c5c1593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:55.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 --2- 172.21.15.123:0/850016093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c1593f0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f1c5c06f2f0 tx=0x7f1c54009840 comp rx=0 tx=0).stop 2023-10-06T04:31:55.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 -- 172.21.15.123:0/850016093 shutdown_connections 2023-10-06T04:31:55.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 --2- 172.21.15.123:0/850016093 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c1593f0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:55.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 --2- 172.21.15.123:0/850016093 >> v2:172.21.15.177:3300/0 conn(0x7f1c5c0688c0 0x7f1c5c0c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:55.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 -- 172.21.15.123:0/850016093 >> 172.21.15.123:0/850016093 conn(0x7f1c5c0c9c90 msgr2=0x7f1c5c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:55.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 -- 172.21.15.123:0/850016093 shutdown_connections 2023-10-06T04:31:55.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.906+0000 7f1c61779700 1 -- 172.21.15.123:0/850016093 wait complete. 2023-10-06T04:31:55.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.907+0000 7f1c61779700 1 Processor -- start 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.907+0000 7f1c61779700 1 -- start start 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c0688c0 0x7f1c5c145970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c5c068c90 0x7f1c5c0c2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c0c2d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c147ed0 con 0x7f1c5c158f70 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c1481b0 con 0x7f1c5c0688c0 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c5c148490 con 0x7f1c5c068c90 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c0c2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c0c2d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33376/0 (socket says 172.21.15.123:33376) 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/2146107419 learned_addr learned my addr 172.21.15.123:0/2146107419 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5affd700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c0688c0 0x7f1c5c145970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:55.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5a7fc700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c5c068c90 0x7f1c5c0c2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c5c068c90 msgr2=0x7f1c5c0c2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c5c068c90 0x7f1c5c0c2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c0688c0 msgr2=0x7f1c5c145970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c0688c0 0x7f1c5c145970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 -- 172.21.15.123:0/2146107419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c54009ae0 con 0x7f1c5c158f70 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c5b7fe700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c0c2d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009ab0 tx=0x7f1c54018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c42ffd700 1 -- 172.21.15.123:0/2146107419 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c54008ac0 con 0x7f1c5c158f70 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c42ffd700 1 -- 172.21.15.123:0/2146107419 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c54008c40 con 0x7f1c5c158f70 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c5c148770 con 0x7f1c5c158f70 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c42ffd700 1 -- 172.21.15.123:0/2146107419 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c5401c410 con 0x7f1c5c158f70 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.908+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c5c145eb0 con 0x7f1c5c158f70 2023-10-06T04:31:55.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.909+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c5c06cc70 con 0x7f1c5c158f70 2023-10-06T04:31:55.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.910+0000 7f1c42ffd700 1 -- 172.21.15.123:0/2146107419 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1c54003040 con 0x7f1c5c158f70 2023-10-06T04:31:55.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.910+0000 7f1c42ffd700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1c48066f30 0x7f1c480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:55.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.910+0000 7f1c5affd700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1c48066f30 0x7f1c480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:55.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.910+0000 7f1c42ffd700 1 -- 172.21.15.123:0/2146107419 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1c54085e80 con 0x7f1c5c158f70 2023-10-06T04:31:55.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.911+0000 7f1c5affd700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1c48066f30 0x7f1c480693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009db0 tx=0x7f1c4c0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:55.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:55.915+0000 7f1c42ffd700 1 -- 172.21.15.123:0/2146107419 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1c54052f90 con 0x7f1c5c158f70 2023-10-06T04:31:56.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.139+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1c5c069f60 con 0x7f1c5c158f70 2023-10-06T04:31:56.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.140+0000 7f1c42ffd700 1 -- 172.21.15.123:0/2146107419 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1c540565b0 con 0x7f1c5c158f70 2023-10-06T04:31:56.142 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:31:56.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1c48066f30 msgr2=0x7f1c480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:56.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1c48066f30 0x7f1c480693e0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f1c4c009db0 tx=0x7f1c4c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:31:56.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 msgr2=0x7f1c5c0c2d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:56.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c0c2d90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f1c54009ab0 tx=0x7f1c54018000 comp rx=0 tx=0).stop 2023-10-06T04:31:56.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 shutdown_connections 2023-10-06T04:31:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1c48066f30 0x7f1c480693e0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1c5c158f70 0x7f1c5c0c2d90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1c5c0688c0 0x7f1c5c145970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 --2- 172.21.15.123:0/2146107419 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1c5c068c90 0x7f1c5c0c2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 >> 172.21.15.123:0/2146107419 conn(0x7f1c5c0c9c90 msgr2=0x7f1c5c1581a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 shutdown_connections 2023-10-06T04:31:56.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:56.142+0000 7f1c61779700 1 -- 172.21.15.123:0/2146107419 wait complete. 2023-10-06T04:31:56.156 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:31:56.156 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:31:56.156 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-10-06T04:31:56.156 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-10-06T04:31:56.156 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-10-06T04:31:56.156 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:31:56.156 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:31:56.157 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:31:56.157 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:31:56.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.157 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-10-06T04:31:56.181 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:56.182 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:56.182 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:56.182 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:56.183 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:56.183 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:56.183 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:56.183 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:56.183 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2023-10-06T04:31:56.183 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:56.183 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42497sec preferred_lft 42497sec 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link 2023-10-06T04:31:56.184 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:56.185 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-10-06T04:31:56.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.185 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:56.185 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:56.185 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add name ceph-brx type bridge 2023-10-06T04:31:56.185 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr flush dev ceph-brx 2023-10-06T04:31:56.185 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set ceph-brx up 2023-10-06T04:31:56.185 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-10-06T04:31:56.185 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:56.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.360 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:31:56.393 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-10-06T04:31:56.394 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.394 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-10-06T04:31:56.487 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-10-06T04:31:56.487 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:31:56.487 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:31:56.487 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:31:56.487 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:31:56.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.488 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:56.488 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:56.488 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:31:56.488 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:31:56.488 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:31:56.488 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:56.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.721 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T04:31:56.739 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.740 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-10-06T04:31:56.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.798 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:56.798 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:56.798 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:31:56.798 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-10-06T04:31:56.798 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:56.893 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-10-06T04:31:56.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-10-06T04:31:56.894 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:57.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:57.097 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:57.098 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:57.098 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.0 up 2023-10-06T04:31:57.098 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.0 master ceph-brx 2023-10-06T04:31:57.098 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:57.195 INFO:tasks.cephfs.fuse_mount:Client client.1 config is {} 2023-10-06T04:31:57.196 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:57.196 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-10-06T04:31:57.214 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-10-06T04:31:57.215 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:57.215 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:31:57.273 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T04:31:57.341 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:57.341 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:57.408 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:57.409 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:57.409 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:57.409 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T04:31:57.468 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:31:57.468 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.1 --id 1) 2023-10-06T04:31:57.510 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T04:31:57.537 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:57.537 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:57.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:2023-10-06T04:31:57.591+0000 7f67796b8540 -1 init, newargv = 0x55d86a7d1a70 newargc=15 2023-10-06T04:31:57.593 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:ceph-fuse[126460]: starting ceph client 2023-10-06T04:31:57.598 INFO:tasks.cephfs.fuse_mount.ceph-fuse.1.smithi177.stderr:ceph-fuse[126460]: starting fuse 2023-10-06T04:31:57.606 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:57.607 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:57.608 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:57.608 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T04:31:57.666 INFO:teuthology.orchestra.run.smithi177.stdout:46 2023-10-06T04:31:57.666 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:31:57.666 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:31:57.666 DEBUG:teuthology.orchestra.run.smithi177:> import glob 2023-10-06T04:31:57.666 DEBUG:teuthology.orchestra.run.smithi177:> import re 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> import os 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> import subprocess 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> def _find_admin_socket(client_name): 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> asok_path = "/var/run/ceph/ceph-client.1.*.asok" 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> files = glob.glob(asok_path) 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> mountpoint = "/home/ubuntu/cephtest/mnt.1" 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> # Given a non-glob path, it better be there 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> if "*" not in asok_path: 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> assert(len(files) == 1) 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> return files[0] 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:57.667 DEBUG:teuthology.orchestra.run.smithi177:> for f in files: 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> contents = proc_f.read() 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> if mountpoint in contents: 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> return f 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> print(_find_admin_socket("client.1")) 2023-10-06T04:31:57.668 DEBUG:teuthology.orchestra.run.smithi177:> ' 2023-10-06T04:31:57.789 INFO:teuthology.orchestra.run.smithi177.stdout:/var/run/ceph/ceph-client.1.126460.asok 2023-10-06T04:31:57.971 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.1.126460.asok 2023-10-06T04:31:57.972 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:57.972 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.1.126460.asok status 2023-10-06T04:31:58.125 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-10-06T04:31:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: "metadata": { 2023-10-06T04:31:58.125 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "entity_id": "1", 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "hostname": "smithi177", 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.1", 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "pid": "126460", 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "root": "/" 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_count": 0, 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_pinned_count": 0, 2023-10-06T04:31:58.126 INFO:teuthology.orchestra.run.smithi177.stdout: "id": 4499, 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "inst": { 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "name": { 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "client", 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "num": 4499 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2838944972 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T04:31:58.127 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 2838944972 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "inst_str": "client.4499 192.168.0.1:0/2838944972", 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "addr_str": "192.168.0.1:0/2838944972", 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "inode_count": 1, 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_epoch": 8, 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch": 23, 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:31:58.128 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklisted": false, 2023-10-06T04:31:58.129 INFO:teuthology.orchestra.run.smithi177.stdout: "fs_name": "cephfs" 2023-10-06T04:31:58.129 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-10-06T04:31:58.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:31:58.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.266+0000 7fa2f5912700 1 Processor -- start 2023-10-06T04:31:58.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.266+0000 7fa2f5912700 1 -- start start 2023-10-06T04:31:58.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2f5912700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f00c98f0 0x7fa2f0159a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:58.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2f5912700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa2f006e2c0 0x7fa2f006e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:58.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2f5912700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa2f015d1f0 con 0x7fa2f0156ea0 2023-10-06T04:31:58.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2f5912700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2f006ec80 con 0x7fa2f006e2c0 2023-10-06T04:31:58.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2f5912700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00c9d40 con 0x7fa2f00c98f0 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2ef7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa2f006e2c0 0x7fa2f006e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2eeffd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa2f0156ea0 0x7fa2f015d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:58126/0 (socket says 172.21.15.123:58126) 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2ef7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa2f006e2c0 0x7fa2f006e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33382/0 (socket says 172.21.15.123:33382) 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2eeffd700 1 -- 172.21.15.123:0/1206338697 learned_addr learned my addr 172.21.15.123:0/1206338697 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2ef7fe700 1 -- 172.21.15.123:0/1206338697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f00c98f0 msgr2=0x7fa2f0159a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2ee7fc700 1 --2- 172.21.15.123:0/1206338697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f00c98f0 0x7fa2f0159a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2ef7fe700 1 --2- 172.21.15.123:0/1206338697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f00c98f0 0x7fa2f0159a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2edffb700 1 -- 172.21.15.123:0/1206338697 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3905038929 0 0) 0x7fa2f015d1f0 con 0x7fa2f0156ea0 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2ef7fe700 1 -- 172.21.15.123:0/1206338697 >> v1:172.21.15.123:6789/0 conn(0x7fa2f0156ea0 legacy=0x7fa2f015d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.267+0000 7fa2ef7fe700 1 -- 172.21.15.123:0/1206338697 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2f015a130 con 0x7fa2f006e2c0 2023-10-06T04:31:58.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2ef7fe700 1 --2- 172.21.15.123:0/1206338697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f006e2c0 0x7fa2f006e740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009e60 tx=0x7fa2e8009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27b0a368dcd9b85 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2edffb700 1 -- 172.21.15.123:0/1206338697 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e800e070 con 0x7fa2f006e2c0 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2edffb700 1 -- 172.21.15.123:0/1206338697 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e800bc30 con 0x7fa2f006e2c0 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2edffb700 1 -- 172.21.15.123:0/1206338697 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e8012430 con 0x7fa2f006e2c0 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 -- 172.21.15.123:0/1206338697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f006e2c0 msgr2=0x7fa2f006e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 --2- 172.21.15.123:0/1206338697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f006e2c0 0x7fa2f006e740 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7fa2e8009e60 tx=0x7fa2e8009780 comp rx=0 tx=0).stop 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 -- 172.21.15.123:0/1206338697 shutdown_connections 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 --2- 172.21.15.123:0/1206338697 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f006e2c0 0x7fa2f006e740 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 --2- 172.21.15.123:0/1206338697 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f00c98f0 0x7fa2f0159a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 -- 172.21.15.123:0/1206338697 >> 172.21.15.123:0/1206338697 conn(0x7fa2f0068770 msgr2=0x7fa2f0068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:58.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 -- 172.21.15.123:0/1206338697 shutdown_connections 2023-10-06T04:31:58.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.268+0000 7fa2f5912700 1 -- 172.21.15.123:0/1206338697 wait complete. 2023-10-06T04:31:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 Processor -- start 2023-10-06T04:31:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 -- start start 2023-10-06T04:31:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f006e2c0 0x7fa2f0144540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f00c98f0 0x7fa2f015fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:58.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f0156ea0 0x7fa2f01600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00b14a0 con 0x7fa2f00c98f0 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00b0b70 con 0x7fa2f0156ea0 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2f5912700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2f00b0e50 con 0x7fa2f006e2c0 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.270+0000 7fa2ee7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f00c98f0 0x7fa2f015fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2eeffd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f006e2c0 0x7fa2f0144540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f00c98f0 0x7fa2f015fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33396/0 (socket says 172.21.15.123:33396) 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 -- 172.21.15.123:0/2817092507 learned_addr learned my addr 172.21.15.123:0/2817092507 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ef7fe700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f0156ea0 0x7fa2f01600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 -- 172.21.15.123:0/2817092507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f006e2c0 msgr2=0x7fa2f0144540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f006e2c0 0x7fa2f0144540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 -- 172.21.15.123:0/2817092507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f0156ea0 msgr2=0x7fa2f01600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f0156ea0 0x7fa2f01600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 -- 172.21.15.123:0/2817092507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2e8009a20 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2ee7fc700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f00c98f0 0x7fa2f015fbb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c7d0 tx=0x7fa2e0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/2817092507 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0011600 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/2817092507 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa2e0011760 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2f00b1130 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/2817092507 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2e0011b20 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.271+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2f0155f10 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.272+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2f006d500 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.273+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/2817092507 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa2e00193f0 con 0x7fa2f00c98f0 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.273+0000 7fa2d6ffd700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa2dc066f30 0x7fa2dc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:31:58.279 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.273+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/2817092507 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa2e00849b0 con 0x7fa2f00c98f0 2023-10-06T04:31:58.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.273+0000 7fa2eeffd700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa2dc066f30 0x7fa2dc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:31:58.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.273+0000 7fa2eeffd700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa2dc066f30 0x7fa2dc0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00cc4b0 tx=0x7fa2e4007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:31:58.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.277+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/2817092507 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa2e0051ac0 con 0x7fa2f00c98f0 2023-10-06T04:31:58.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.504+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa2f00b1830 con 0x7fa2f00c98f0 2023-10-06T04:31:58.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.504+0000 7fa2d6ffd700 1 -- 172.21.15.123:0/2817092507 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa2e00550e0 con 0x7fa2f00c98f0 2023-10-06T04:31:58.507 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa2dc066f30 msgr2=0x7fa2dc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa2dc066f30 0x7fa2dc0693e0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fa2f00cc4b0 tx=0x7fa2e4007d30 comp rx=0 tx=0).stop 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f00c98f0 msgr2=0x7fa2f015fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f00c98f0 0x7fa2f015fbb0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fa2e000c7d0 tx=0x7fa2e0008040 comp rx=0 tx=0).stop 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 shutdown_connections 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa2dc066f30 0x7fa2dc0693e0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa2f0156ea0 0x7fa2f01600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa2f006e2c0 0x7fa2f0144540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 --2- 172.21.15.123:0/2817092507 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa2f00c98f0 0x7fa2f015fbb0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:31:58.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 >> 172.21.15.123:0/2817092507 conn(0x7fa2f0068770 msgr2=0x7fa2f00c75d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:31:58.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 shutdown_connections 2023-10-06T04:31:58.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:31:58.506+0000 7fa2f5912700 1 -- 172.21.15.123:0/2817092507 wait complete. 2023-10-06T04:31:58.520 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:31:58.520 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:31:58.521 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi177.front.sepia.ceph.com 2023-10-06T04:31:58.521 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-10-06T04:31:58.521 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-10-06T04:31:58.521 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:31:58.521 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:31:58.521 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:31:58.521 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-10-06T04:31:58.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.521 DEBUG:teuthology.orchestra.run.smithi177:> ip addr 2023-10-06T04:31:58.541 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:40 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f0 2023-10-06T04:31:58.542 INFO:teuthology.orchestra.run.smithi177.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:d9:19:41 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: altname enp7s0f1 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4a brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 0c:c4:7a:ea:b0:4b brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: inet 172.21.15.177/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft 42495sec preferred_lft 42495sec 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::ec4:7aff:feea:b04b/64 scope link 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.543 INFO:teuthology.orchestra.run.smithi177.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 82:02:c0:5b:b0:68 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::f4db:7dff:fe9c:b8ca/64 scope link 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 82:02:c0:5b:b0:68 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::8002:c0ff:fe5b:b068/64 scope link 2023-10-06T04:31:58.544 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.545 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.545 DEBUG:teuthology.orchestra.run.smithi177:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:31:58.610 INFO:teuthology.orchestra.run.smithi177.stdout:1 2023-10-06T04:31:58.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.611 DEBUG:teuthology.orchestra.run.smithi177:> route 2023-10-06T04:31:58.673 INFO:teuthology.orchestra.run.smithi177.stdout:Kernel IP routing table 2023-10-06T04:31:58.674 INFO:teuthology.orchestra.run.smithi177.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:31:58.674 INFO:teuthology.orchestra.run.smithi177.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:31:58.674 INFO:teuthology.orchestra.run.smithi177.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:31:58.674 INFO:teuthology.orchestra.run.smithi177.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:31:58.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.674 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:58.674 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:58.675 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:31:58.675 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:31:58.675 DEBUG:teuthology.orchestra.run.smithi177:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:31:58.675 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:58.780 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.780 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T04:31:58.798 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:31:58.798 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.799 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list-id 2023-10-06T04:31:58.856 INFO:teuthology.orchestra.run.smithi177.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-10-06T04:31:58.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.856 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:58.856 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:58.856 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:31:58.856 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-10-06T04:31:58.857 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:58.950 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:58.950 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:31:58.987 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 8e:43:ed:ec:53:da brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:31:58.988 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:31:58.988 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:58.988 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::8c43:edff:feec:53da/64 scope link 2023-10-06T04:31:58.988 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:59.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:59.005 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-10-06T04:31:59.081 INFO:teuthology.orchestra.run.smithi177.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:31:59.081 INFO:teuthology.orchestra.run.smithi177.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:31:59.081 INFO:teuthology.orchestra.run.smithi177.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:31:59.081 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:59.081 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 ::1/128 scope host 2023-10-06T04:31:59.081 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:59.081 INFO:teuthology.orchestra.run.smithi177.stdout:2: veth0@if7: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:31:59.082 INFO:teuthology.orchestra.run.smithi177.stdout: link/ether 8e:43:ed:ec:53:da brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-10-06T04:31:59.082 INFO:teuthology.orchestra.run.smithi177.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-10-06T04:31:59.082 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:59.082 INFO:teuthology.orchestra.run.smithi177.stdout: inet6 fe80::8c43:edff:feec:53da/64 scope link 2023-10-06T04:31:59.082 INFO:teuthology.orchestra.run.smithi177.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:31:59.090 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-10-06T04:31:59.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:59.090 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:59.091 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:59.091 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-10-06T04:31:59.091 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-10-06T04:31:59.091 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-10-06T04:31:59.091 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-10-06T04:31:59.091 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-10-06T04:31:59.091 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:59.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:31:59.313 DEBUG:teuthology.orchestra.run.smithi177:> (cd / && exec bash -c ' 2023-10-06T04:31:59.314 DEBUG:teuthology.orchestra.run.smithi177:> set -e 2023-10-06T04:31:59.314 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set brx.1 up 2023-10-06T04:31:59.314 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link set dev brx.1 master ceph-brx 2023-10-06T04:31:59.314 DEBUG:teuthology.orchestra.run.smithi177:> ') 2023-10-06T04:31:59.411 INFO:tasks.cephfs.fuse_mount:Client client.3 config is {} 2023-10-06T04:31:59.412 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:59.412 DEBUG:teuthology.orchestra.run.smithi177:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-10-06T04:31:59.430 INFO:teuthology.orchestra.run.smithi177.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-10-06T04:31:59.430 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:31:59.430 DEBUG:teuthology.orchestra.run.smithi177:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-10-06T04:31:59.488 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T04:31:59.555 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:59.555 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:59.623 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:59.623 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:59.624 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:59.624 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T04:31:59.683 INFO:teuthology.orchestra.run.smithi177.stdout:46 2023-10-06T04:31:59.683 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [46] 2023-10-06T04:31:59.683 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.3 --id 3) 2023-10-06T04:31:59.725 DEBUG:teuthology.orchestra.run.smithi177:> sudo modprobe fuse 2023-10-06T04:31:59.752 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:31:59.753 DEBUG:teuthology.orchestra.run.smithi177:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:31:59.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi177.stderr:2023-10-06T04:31:59.807+0000 7fd68a60f540 -1 init, newargv = 0x56001e9af360 newargc=15 2023-10-06T04:31:59.809 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi177.stderr:ceph-fuse[127135]: starting ceph client 2023-10-06T04:31:59.813 INFO:tasks.cephfs.fuse_mount.ceph-fuse.3.smithi177.stderr:ceph-fuse[127135]: starting fuse 2023-10-06T04:31:59.821 INFO:teuthology.orchestra.run.smithi177.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:31:59.823 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:31:59.823 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:31:59.823 DEBUG:teuthology.orchestra.run.smithi177:> ls /sys/fs/fuse/connections 2023-10-06T04:31:59.882 INFO:teuthology.orchestra.run.smithi177.stdout:46 2023-10-06T04:31:59.882 INFO:teuthology.orchestra.run.smithi177.stdout:48 2023-10-06T04:31:59.882 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46, 48] 2023-10-06T04:31:59.882 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:31:59.882 DEBUG:teuthology.orchestra.run.smithi177:> import glob 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> import re 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> import os 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> import subprocess 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> def _find_admin_socket(client_name): 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> asok_path = "/var/run/ceph/ceph-client.3.*.asok" 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> files = glob.glob(asok_path) 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> mountpoint = "/home/ubuntu/cephtest/mnt.3" 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> # Given a non-glob path, it better be there 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> if "*" not in asok_path: 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> assert(len(files) == 1) 2023-10-06T04:31:59.883 DEBUG:teuthology.orchestra.run.smithi177:> return files[0] 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> for f in files: 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> contents = proc_f.read() 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> if mountpoint in contents: 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> return f 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> print(_find_admin_socket("client.3")) 2023-10-06T04:31:59.884 DEBUG:teuthology.orchestra.run.smithi177:> ' 2023-10-06T04:32:00.005 INFO:teuthology.orchestra.run.smithi177.stdout:/var/run/ceph/ceph-client.3.127135.asok 2023-10-06T04:32:00.187 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.3.127135.asok 2023-10-06T04:32:00.187 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.187 DEBUG:teuthology.orchestra.run.smithi177:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.3.127135.asok status 2023-10-06T04:32:00.340 INFO:teuthology.orchestra.run.smithi177.stdout:{ 2023-10-06T04:32:00.340 INFO:teuthology.orchestra.run.smithi177.stdout: "metadata": { 2023-10-06T04:32:00.340 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T04:32:00.340 INFO:teuthology.orchestra.run.smithi177.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T04:32:00.340 INFO:teuthology.orchestra.run.smithi177.stdout: "entity_id": "3", 2023-10-06T04:32:00.340 INFO:teuthology.orchestra.run.smithi177.stdout: "hostname": "smithi177", 2023-10-06T04:32:00.340 INFO:teuthology.orchestra.run.smithi177.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.3", 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "pid": "127135", 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "root": "/" 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_count": 0, 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "dentry_pinned_count": 0, 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "id": 4505, 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "inst": { 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "name": { 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "client", 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "num": 4505 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:32:00.341 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3973931861 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: } 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": { 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "type": "v1", 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "addr": "192.168.0.2:0", 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "nonce": 3973931861 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: }, 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "inst_str": "client.4505 192.168.0.2:0/3973931861", 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "addr_str": "192.168.0.2:0/3973931861", 2023-10-06T04:32:00.342 INFO:teuthology.orchestra.run.smithi177.stdout: "inode_count": 1, 2023-10-06T04:32:00.343 INFO:teuthology.orchestra.run.smithi177.stdout: "mds_epoch": 8, 2023-10-06T04:32:00.343 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch": 23, 2023-10-06T04:32:00.343 INFO:teuthology.orchestra.run.smithi177.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:32:00.343 INFO:teuthology.orchestra.run.smithi177.stdout: "blocklisted": false, 2023-10-06T04:32:00.343 INFO:teuthology.orchestra.run.smithi177.stdout: "fs_name": "cephfs" 2023-10-06T04:32:00.343 INFO:teuthology.orchestra.run.smithi177.stdout:} 2023-10-06T04:32:00.350 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.350 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2023-10-06T04:32:00.351 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:00.374 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2023-10-06T04:32:00.374 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:00.374 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.374 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:00.450 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.450 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2023-10-06T04:32:00.450 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.2 2023-10-06T04:32:00.515 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2023-10-06T04:32:00.515 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.2 2023-10-06T04:32:00.515 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.515 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-10-06T04:32:00.589 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.589 DEBUG:teuthology.orchestra.run.smithi177:> stat --file-system '--printf=%T 2023-10-06T04:32:00.589 DEBUG:teuthology.orchestra.run.smithi177:> ' -- /home/ubuntu/cephtest/mnt.1 2023-10-06T04:32:00.608 INFO:teuthology.orchestra.run.smithi177.stdout:fuseblk 2023-10-06T04:32:00.608 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.1 2023-10-06T04:32:00.609 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.609 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-10-06T04:32:00.676 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.676 DEBUG:teuthology.orchestra.run.smithi177:> stat --file-system '--printf=%T 2023-10-06T04:32:00.676 DEBUG:teuthology.orchestra.run.smithi177:> ' -- /home/ubuntu/cephtest/mnt.3 2023-10-06T04:32:00.734 INFO:teuthology.orchestra.run.smithi177.stdout:fuseblk 2023-10-06T04:32:00.734 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.3 2023-10-06T04:32:00.735 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:00.735 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-10-06T04:32:00.803 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-10-06T04:32:00.827 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:00.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.965+0000 7ff1af4b0700 1 Processor -- start 2023-10-06T04:32:00.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.966+0000 7ff1af4b0700 1 -- start start 2023-10-06T04:32:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.966+0000 7ff1af4b0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff1a8067660 0x7ff1a8067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.966+0000 7ff1af4b0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a806f070 con 0x7ff1a80c9570 2023-10-06T04:32:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.966+0000 7ff1af4b0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1a8067f70 con 0x7ff1a80aa950 2023-10-06T04:32:00.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.966+0000 7ff1af4b0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1a806a750 con 0x7ff1a8067660 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff1ad24c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff1a8067660 0x7ff1a8067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff1aca4b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff1a80c9570 0x7ff1a806e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53082/0 (socket says 172.21.15.123:53082) 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff1aca4b700 1 -- 172.21.15.123:0/1122741119 learned_addr learned my addr 172.21.15.123:0/1122741119 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff1ad24c700 1 -- 172.21.15.123:0/1122741119 >> v1:172.21.15.177:6789/0 conn(0x7ff1a80aa950 legacy=0x7ff1a806e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff1ad24c700 1 -- 172.21.15.123:0/1122741119 >> v1:172.21.15.123:6789/0 conn(0x7ff1a80c9570 legacy=0x7ff1a806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff1ad24c700 1 -- 172.21.15.123:0/1122741119 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1a806aa30 con 0x7ff1a8067660 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff19ffff700 1 -- 172.21.15.123:0/1122741119 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4020746901 0 0) 0x7ff1a806f070 con 0x7ff1a80c9570 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff1ad24c700 1 --2- 172.21.15.123:0/1122741119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a8067660 0x7ff1a8067a30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1980098e0 tx=0x7ff1980091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7f228b816d9bda7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff19ffff700 1 -- 172.21.15.123:0/1122741119 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff198009e80 con 0x7ff1a8067660 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff19ffff700 1 -- 172.21.15.123:0/1122741119 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff19800b860 con 0x7ff1a8067660 2023-10-06T04:32:00.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.967+0000 7ff19ffff700 1 -- 172.21.15.123:0/1122741119 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff19800bbe0 con 0x7ff1a8067660 2023-10-06T04:32:00.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.968+0000 7ff1af4b0700 1 -- 172.21.15.123:0/1122741119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a8067660 msgr2=0x7ff1a8067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:00.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.968+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/1122741119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a8067660 0x7ff1a8067a30 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1980098e0 tx=0x7ff1980091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:00.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.968+0000 7ff1af4b0700 1 -- 172.21.15.123:0/1122741119 shutdown_connections 2023-10-06T04:32:00.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.968+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/1122741119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a8067660 0x7ff1a8067a30 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:00.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.968+0000 7ff1af4b0700 1 -- 172.21.15.123:0/1122741119 >> 172.21.15.123:0/1122741119 conn(0x7ff1a80cda80 msgr2=0x7ff1a80c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:00.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.968+0000 7ff1af4b0700 1 -- 172.21.15.123:0/1122741119 shutdown_connections 2023-10-06T04:32:00.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.968+0000 7ff1af4b0700 1 -- 172.21.15.123:0/1122741119 wait complete. 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.969+0000 7ff1af4b0700 1 Processor -- start 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.969+0000 7ff1af4b0700 1 -- start start 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1af4b0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1a8067660 0x7ff1a816ac90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1af4b0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a80aa950 0x7ff1a816c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1af4b0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a80c9570 0x7ff1a8157840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1af4b0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a815df50 con 0x7ff1a80aa950 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1af4b0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a81541b0 con 0x7ff1a80c9570 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1af4b0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1a816b1d0 con 0x7ff1a8067660 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1aca4b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a80aa950 0x7ff1a816c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1aca4b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a80aa950 0x7ff1a816c7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41996/0 (socket says 172.21.15.123:41996) 2023-10-06T04:32:00.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1aca4b700 1 -- 172.21.15.123:0/446680520 learned_addr learned my addr 172.21.15.123:0/446680520 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1ad24c700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1a8067660 0x7ff1a816ac90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.970+0000 7ff1ada4d700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a80c9570 0x7ff1a8157840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1aca4b700 1 -- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1a8067660 msgr2=0x7ff1a816ac90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1aca4b700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1a8067660 0x7ff1a816ac90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1aca4b700 1 -- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a80c9570 msgr2=0x7ff1a8157840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1aca4b700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a80c9570 0x7ff1a8157840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1aca4b700 1 -- 172.21.15.123:0/446680520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff198009420 con 0x7ff1a80aa950 2023-10-06T04:32:00.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1ad24c700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1a8067660 0x7ff1a816ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1aca4b700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a80aa950 0x7ff1a816c7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c590 tx=0x7ff1a4000fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff19d7fa700 1 -- 172.21.15.123:0/446680520 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4005b30 con 0x7ff1a80aa950 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff19d7fa700 1 -- 172.21.15.123:0/446680520 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1a4005c90 con 0x7ff1a80aa950 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1a815d990 con 0x7ff1a80aa950 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff19d7fa700 1 -- 172.21.15.123:0/446680520 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1a4007320 con 0x7ff1a80aa950 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.971+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1a8154440 con 0x7ff1a80aa950 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.972+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1a80cde80 con 0x7ff1a80aa950 2023-10-06T04:32:00.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.972+0000 7ff19d7fa700 1 -- 172.21.15.123:0/446680520 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff1a401b3b0 con 0x7ff1a80aa950 2023-10-06T04:32:00.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.973+0000 7ff19d7fa700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff18c066f30 0x7ff18c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:00.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.973+0000 7ff1ad24c700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff18c066f30 0x7ff18c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:00.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.973+0000 7ff19d7fa700 1 -- 172.21.15.123:0/446680520 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff1a4083b40 con 0x7ff1a80aa950 2023-10-06T04:32:00.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.973+0000 7ff1ad24c700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff18c066f30 0x7ff18c0693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1980093f0 tx=0x7ff198017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:00.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:00.977+0000 7ff19d7fa700 1 -- 172.21.15.123:0/446680520 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff1a4050c50 con 0x7ff1a80aa950 2023-10-06T04:32:01.166 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.164+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff1a804bb80 con 0x7ff1a80aa950 2023-10-06T04:32:01.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.165+0000 7ff19d7fa700 1 -- 172.21.15.123:0/446680520 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7ff1a4017090 con 0x7ff1a80aa950 2023-10-06T04:32:01.167 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:01.167 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:01.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff18c066f30 msgr2=0x7ff18c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff18c066f30 0x7ff18c0693e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff1980093f0 tx=0x7ff198017000 comp rx=0 tx=0).stop 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a80aa950 msgr2=0x7ff1a816c7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a80aa950 0x7ff1a816c7a0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ff1a400c590 tx=0x7ff1a4000fe0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 shutdown_connections 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff18c066f30 0x7ff18c0693e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1a80c9570 0x7ff1a8157840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1a8067660 0x7ff1a816ac90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 --2- 172.21.15.123:0/446680520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff1a80aa950 0x7ff1a816c7a0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 >> 172.21.15.123:0/446680520 conn(0x7ff1a80cda80 msgr2=0x7ff1a8158a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 shutdown_connections 2023-10-06T04:32:01.170 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.167+0000 7ff1af4b0700 1 -- 172.21.15.123:0/446680520 wait complete. 2023-10-06T04:32:01.181 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:31:32.045143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:31:33.390421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:32:01.181 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:01.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.300+0000 7fcb9efb1700 1 Processor -- start 2023-10-06T04:32:01.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.300+0000 7fcb9efb1700 1 -- start start 2023-10-06T04:32:01.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9efb1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb98159400 0x7fcb980c77d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9efb1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fcb981597d0 0x7fcb98068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9efb1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb980c2db0 con 0x7fcb980c6de0 2023-10-06T04:32:01.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9efb1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb980c8090 con 0x7fcb98159400 2023-10-06T04:32:01.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9efb1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb9806aa50 con 0x7fcb981597d0 2023-10-06T04:32:01.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9cd4d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb98159400 0x7fcb980c77d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9cd4d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcb98159400 0x7fcb980c77d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42004/0 (socket says 172.21.15.123:42004) 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb9cd4d700 1 -- 172.21.15.123:0/412877644 learned_addr learned my addr 172.21.15.123:0/412877644 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.301+0000 7fcb97fff700 1 --2- 172.21.15.123:0/412877644 >> v2:172.21.15.177:3300/0 conn(0x7fcb981597d0 0x7fcb98068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9cd4d700 1 -- 172.21.15.123:0/412877644 >> v2:172.21.15.177:3300/0 conn(0x7fcb981597d0 msgr2=0x7fcb98068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9cd4d700 1 --2- 172.21.15.123:0/412877644 >> v2:172.21.15.177:3300/0 conn(0x7fcb981597d0 0x7fcb98068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9cd4d700 1 -- 172.21.15.123:0/412877644 >> v1:172.21.15.177:6789/0 conn(0x7fcb980c6de0 legacy=0x7fcb980c1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9cd4d700 1 -- 172.21.15.123:0/412877644 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb980c7db0 con 0x7fcb98159400 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9cd4d700 1 --2- 172.21.15.123:0/412877644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c77d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb9806ba10 tx=0x7fcb8c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2375dd3d1048d24e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb977fe700 1 -- 172.21.15.123:0/412877644 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00b860 con 0x7fcb98159400 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb977fe700 1 -- 172.21.15.123:0/412877644 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb8c00b9c0 con 0x7fcb98159400 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb977fe700 1 -- 172.21.15.123:0/412877644 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb8c00bd60 con 0x7fcb98159400 2023-10-06T04:32:01.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 -- 172.21.15.123:0/412877644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 msgr2=0x7fcb980c77d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/412877644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c77d0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fcb9806ba10 tx=0x7fcb8c009250 comp rx=0 tx=0).stop 2023-10-06T04:32:01.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 -- 172.21.15.123:0/412877644 shutdown_connections 2023-10-06T04:32:01.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/412877644 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c77d0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/412877644 >> v2:172.21.15.177:3300/0 conn(0x7fcb981597d0 0x7fcb98068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 -- 172.21.15.123:0/412877644 >> 172.21.15.123:0/412877644 conn(0x7fcb980c9ce0 msgr2=0x7fcb9806daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:01.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 -- 172.21.15.123:0/412877644 shutdown_connections 2023-10-06T04:32:01.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.302+0000 7fcb9efb1700 1 -- 172.21.15.123:0/412877644 wait complete. 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 Processor -- start 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 -- start start 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb980c6de0 0x7fcb98152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb981597d0 0x7fcb980c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb98153450 con 0x7fcb98159400 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb9815ef40 con 0x7fcb981597d0 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.304+0000 7fcb9efb1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb980c43c0 con 0x7fcb980c6de0 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42016/0 (socket says 172.21.15.123:42016) 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 -- 172.21.15.123:0/3625197844 learned_addr learned my addr 172.21.15.123:0/3625197844 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb9d54e700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb981597d0 0x7fcb980c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb9cd4d700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb980c6de0 0x7fcb98152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 -- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb980c6de0 msgr2=0x7fcb98152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb980c6de0 0x7fcb98152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 -- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb981597d0 msgr2=0x7fcb980c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb981597d0 0x7fcb980c3d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 -- 172.21.15.123:0/3625197844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb8c0094f0 con 0x7fcb98159400 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb97fff700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c3850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400d890 tx=0x7fcb84008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb94ff9700 1 -- 172.21.15.123:0/3625197844 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb840125f0 con 0x7fcb98159400 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb94ff9700 1 -- 172.21.15.123:0/3625197844 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb84012770 con 0x7fcb98159400 2023-10-06T04:32:01.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb94ff9700 1 -- 172.21.15.123:0/3625197844 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb84012b30 con 0x7fcb98159400 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb980c46a0 con 0x7fcb98159400 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.305+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb980c4c60 con 0x7fcb98159400 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.306+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb9806c720 con 0x7fcb98159400 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.306+0000 7fcb94ff9700 1 -- 172.21.15.123:0/3625197844 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcb8401e460 con 0x7fcb98159400 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.307+0000 7fcb94ff9700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcb7c066f30 0x7fcb7c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.307+0000 7fcb94ff9700 1 -- 172.21.15.123:0/3625197844 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcb84084c30 con 0x7fcb98159400 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.307+0000 7fcb9cd4d700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcb7c066f30 0x7fcb7c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.307+0000 7fcb9cd4d700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcb7c066f30 0x7fcb7c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcb9806ba10 tx=0x7fcb8c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.311+0000 7fcb94ff9700 1 -- 172.21.15.123:0/3625197844 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcb84051d40 con 0x7fcb98159400 2023-10-06T04:32:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.494+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcb9804bb80 con 0x7fcb98159400 2023-10-06T04:32:01.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.494+0000 7fcb94ff9700 1 -- 172.21.15.123:0/3625197844 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcb84055360 con 0x7fcb98159400 2023-10-06T04:32:01.497 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcb7c066f30 msgr2=0x7fcb7c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcb7c066f30 0x7fcb7c0693e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7fcb9806ba10 tx=0x7fcb8c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 msgr2=0x7fcb980c3850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c3850 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fcb8400d890 tx=0x7fcb84008040 comp rx=0 tx=0).stop 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 shutdown_connections 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcb7c066f30 0x7fcb7c0693e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcb981597d0 0x7fcb980c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcb980c6de0 0x7fcb98152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.499 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 --2- 172.21.15.123:0/3625197844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcb98159400 0x7fcb980c3850 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 >> 172.21.15.123:0/3625197844 conn(0x7fcb980c9ce0 msgr2=0x7fcb9806ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:01.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 shutdown_connections 2023-10-06T04:32:01.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.496+0000 7fcb9efb1700 1 -- 172.21.15.123:0/3625197844 wait complete. 2023-10-06T04:32:01.510 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:01.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.627+0000 7fe6a6020700 1 Processor -- start 2023-10-06T04:32:01.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.627+0000 7fe6a6020700 1 -- start start 2023-10-06T04:32:01.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.628+0000 7fe6a6020700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe6a00c8a10 0x7fe6a006e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.628+0000 7fe6a6020700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a006a4e0 con 0x7fe6a013fa70 2023-10-06T04:32:01.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.628+0000 7fe6a6020700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6a00c8ec0 con 0x7fe6a00cabf0 2023-10-06T04:32:01.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.628+0000 7fe6a6020700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6a00c91a0 con 0x7fe6a00c8a10 2023-10-06T04:32:01.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.628+0000 7fe69f7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe6a013fa70 0x7fe6a00c8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53098/0 (socket says 172.21.15.123:53098) 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.628+0000 7fe69f7fe700 1 -- 172.21.15.123:0/1750537657 learned_addr learned my addr 172.21.15.123:0/1750537657 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3690924995 0 0) 0x7fe6a006a4e0 con 0x7fe6a013fa70 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe68c003570 con 0x7fe6a013fa70 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69effd700 1 --2- 172.21.15.123:0/1750537657 >> v2:172.21.15.177:3300/0 conn(0x7fe6a00c8a10 0x7fe6a006e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3627852237 0 0) 0x7fe68c003570 con 0x7fe6a013fa70 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 >> v2:172.21.15.177:3300/0 conn(0x7fe6a00c8a10 msgr2=0x7fe6a006e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 --2- 172.21.15.123:0/1750537657 >> v2:172.21.15.177:3300/0 conn(0x7fe6a00c8a10 0x7fe6a006e2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 >> v1:172.21.15.177:6789/0 conn(0x7fe6a00cabf0 legacy=0x7fe6a0159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00c9480 con 0x7fe6a013fa70 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fe688001470 con 0x7fe6a013fa70 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 >> v1:172.21.15.123:6789/0 conn(0x7fe6a013fa70 legacy=0x7fe6a00c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c003a60 0x7fe68c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c004810 0x7fe68c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe68c0051c0 0x7fe68c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00c91a0 con 0x7fe68c004810 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c00c0d0 con 0x7fe68c0051c0 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe68c00c290 con 0x7fe68c003a60 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69effd700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c004810 0x7fe68c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69f7fe700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c003a60 0x7fe68c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69ffff700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe68c0051c0 0x7fe68c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69effd700 1 -- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c003a60 msgr2=0x7fe68c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69effd700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c003a60 0x7fe68c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69effd700 1 -- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe68c0051c0 msgr2=0x7fe68c00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:01.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69effd700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe68c0051c0 0x7fe68c00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69ffff700 1 -- 172.21.15.123:0/1750537657 reap_dead start 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.629+0000 7fe69effd700 1 -- 172.21.15.123:0/1750537657 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe68c003570 con 0x7fe68c004810 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe69effd700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c004810 0x7fe68c004c80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe690009c00 tx=0x7fe6900091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1389f6533f1852b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69000e710 con 0x7fe68c004810 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe69000e870 con 0x7fe68c004810 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe69e7fc700 1 -- 172.21.15.123:0/1750537657 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe69000ec30 con 0x7fe68c004810 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe6a6020700 1 -- 172.21.15.123:0/1750537657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c004810 msgr2=0x7fe68c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe6a6020700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c004810 0x7fe68c004c80 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe690009c00 tx=0x7fe6900091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe6a6020700 1 -- 172.21.15.123:0/1750537657 shutdown_connections 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe6a6020700 1 --2- 172.21.15.123:0/1750537657 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe68c004810 0x7fe68c004c80 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe6a6020700 1 -- 172.21.15.123:0/1750537657 >> 172.21.15.123:0/1750537657 conn(0x7fe6a00cd020 msgr2=0x7fe6a00cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:01.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe6a6020700 1 -- 172.21.15.123:0/1750537657 shutdown_connections 2023-10-06T04:32:01.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.630+0000 7fe6a6020700 1 -- 172.21.15.123:0/1750537657 wait complete. 2023-10-06T04:32:01.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 Processor -- start 2023-10-06T04:32:01.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 -- start start 2023-10-06T04:32:01.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c004810 0x7fe6a013ad50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a00b07a0 0x7fe6a00b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a00b1cd0 0x7fe6a00b2170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00b26b0 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00ac940 con 0x7fe6a00b07a0 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.632+0000 7fe6a6020700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6a00b12b0 con 0x7fe68c004810 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a00b1cd0 0x7fe6a00b2170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a00b1cd0 0x7fe6a00b2170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42030/0 (socket says 172.21.15.123:42030) 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 -- 172.21.15.123:0/3289033698 learned_addr learned my addr 172.21.15.123:0/3289033698 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69effd700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a00b07a0 0x7fe6a00b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69f7fe700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c004810 0x7fe6a013ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 -- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c004810 msgr2=0x7fe6a013ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c004810 0x7fe6a013ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 -- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a00b07a0 msgr2=0x7fe6a00b0c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a00b07a0 0x7fe6a00b0c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 -- 172.21.15.123:0/3289033698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6a00b1590 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69effd700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a00b07a0 0x7fe6a00b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69f7fe700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c004810 0x7fe6a013ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe69ffff700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a00b1cd0 0x7fe6a00b2170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe6a006e270 tx=0x7fe698001640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe6977fe700 1 -- 172.21.15.123:0/3289033698 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698006e80 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe6977fe700 1 -- 172.21.15.123:0/3289033698 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe698014480 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6a00b1870 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe6977fe700 1 -- 172.21.15.123:0/3289033698 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe698014840 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.633+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6a013b290 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.634+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6a00ac710 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.634+0000 7fe6977fe700 1 -- 172.21.15.123:0/3289033698 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe698007070 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.635+0000 7fe6977fe700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe680066f30 0x7fe6800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.642 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.635+0000 7fe6977fe700 1 -- 172.21.15.123:0/3289033698 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe6980843b0 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.642 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.635+0000 7fe69f7fe700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe680066f30 0x7fe6800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.642 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.635+0000 7fe69f7fe700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe680066f30 0x7fe6800693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0155c10 tx=0x7fe688009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.642 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.639+0000 7fe6977fe700 1 -- 172.21.15.123:0/3289033698 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe6a00ac710 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.819+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fe6a00ac710 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.819+0000 7fe6977fe700 1 -- 172.21.15.123:0/3289033698 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fe6a00ac710 con 0x7fe6a00b1cd0 2023-10-06T04:32:01.822 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe680066f30 msgr2=0x7fe6800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe680066f30 0x7fe6800693e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fe6a0155c10 tx=0x7fe688009040 comp rx=0 tx=0).stop 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a00b1cd0 msgr2=0x7fe6a00b2170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a00b1cd0 0x7fe6a00b2170 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fe6a006e270 tx=0x7fe698001640 comp rx=0 tx=0).stop 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 shutdown_connections 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe680066f30 0x7fe6800693e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe6a00b1cd0 0x7fe6a00b2170 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe68c004810 0x7fe6a013ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 --2- 172.21.15.123:0/3289033698 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe6a00b07a0 0x7fe6a00b0c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 >> 172.21.15.123:0/3289033698 conn(0x7fe6a00cd020 msgr2=0x7fe6a00c82c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:01.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 shutdown_connections 2023-10-06T04:32:01.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.821+0000 7fe6a6020700 1 -- 172.21.15.123:0/3289033698 wait complete. 2023-10-06T04:32:01.835 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:32:01.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.952+0000 7f03cb1dc700 1 Processor -- start 2023-10-06T04:32:01.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.953+0000 7f03cb1dc700 1 -- start start 2023-10-06T04:32:01.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.953+0000 7f03cb1dc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 0x7f03c4161eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.953+0000 7f03cb1dc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c40c7810 con 0x7f03c4157450 2023-10-06T04:32:01.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.953+0000 7f03cb1dc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03c40ac9f0 con 0x7f03c4157080 2023-10-06T04:32:01.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.953+0000 7f03cb1dc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c40c6ee0 con 0x7f03c4068d80 2023-10-06T04:32:01.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.953+0000 7f03c3fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f03c4157450 0x7f03c4068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53112/0 (socket says 172.21.15.123:53112) 2023-10-06T04:32:01.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.953+0000 7f03c3fff700 1 -- 172.21.15.123:0/4013594028 learned_addr learned my addr 172.21.15.123:0/4013594028 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:01.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c9779700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 0x7f03c4161eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2351595795 0 0) 0x7f03c40c7810 con 0x7f03c4157450 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03b4003570 con 0x7f03c4157450 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2200815146 0 0) 0x7f03c40ac9f0 con 0x7f03c4157080 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03c40c7810 con 0x7f03c4157080 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1334971390 0 0) 0x7f03b4003570 con 0x7f03c4157450 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 msgr2=0x7f03c4161eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 0x7f03c4161eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 >> v1:172.21.15.177:6789/0 conn(0x7f03c4157080 legacy=0x7f03c40abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c40c71c0 con 0x7f03c4157450 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f03b0001470 con 0x7f03c4157450 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 >> v1:172.21.15.123:6789/0 conn(0x7f03c4157450 legacy=0x7f03c4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 0x7f03b4003c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b4004180 0x7f03b4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03b4004d50 0x7f03b40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c40c6ee0 con 0x7f03b4004180 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b400bba0 con 0x7f03b4004d50 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b400bd60 con 0x7f03c4068d80 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c3fff700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b4004180 0x7f03b4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c8f78700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 0x7f03b4003c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c9779700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03b4004d50 0x7f03b40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c3fff700 1 -- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 msgr2=0x7f03b4003c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c3fff700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 0x7f03b4003c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c3fff700 1 -- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03b4004d50 msgr2=0x7f03b40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:01.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c3fff700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03b4004d50 0x7f03b40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.954+0000 7f03c3fff700 1 -- 172.21.15.123:0/4013594028 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b4003570 con 0x7f03b4004180 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03c8f78700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03c4068d80 0x7f03b4003c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03c3fff700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b4004180 0x7f03b4004810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c2d0 tx=0x7f03b0001e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bc7bfe29ef0b9e5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b000cba0 con 0x7f03b4004180 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b000cd00 con 0x7f03b4004180 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03c37fe700 1 -- 172.21.15.123:0/4013594028 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0005b70 con 0x7f03b4004180 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03cb1dc700 1 -- 172.21.15.123:0/4013594028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b4004180 msgr2=0x7f03b4004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03cb1dc700 1 --2- 172.21.15.123:0/4013594028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03b4004180 0x7f03b4004810 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c2d0 tx=0x7f03b0001e10 comp rx=0 tx=0).stop 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03c9779700 1 -- 172.21.15.123:0/4013594028 reap_dead start 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03cb1dc700 1 -- 172.21.15.123:0/4013594028 shutdown_connections 2023-10-06T04:32:01.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03cb1dc700 1 -- 172.21.15.123:0/4013594028 >> 172.21.15.123:0/4013594028 conn(0x7f03c40c9ca0 msgr2=0x7f03c406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:01.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03cb1dc700 1 -- 172.21.15.123:0/4013594028 shutdown_connections 2023-10-06T04:32:01.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.955+0000 7f03cb1dc700 1 -- 172.21.15.123:0/4013594028 wait complete. 2023-10-06T04:32:01.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.956+0000 7f03cb1dc700 1 Processor -- start 2023-10-06T04:32:01.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03cb1dc700 1 -- start start 2023-10-06T04:32:01.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03cb1dc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03b4004180 0x7f03c4149fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03cb1dc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c414b7e0 0x7f03c414bc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03cb1dc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c40b0b30 0x7f03c40b0fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03cb1dc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c414c190 con 0x7f03c414b7e0 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03cb1dc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03c41562b0 con 0x7f03c40b0b30 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03cb1dc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03c41556a0 con 0x7f03b4004180 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c414b7e0 0x7f03c414bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03c3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c414b7e0 0x7f03c414bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42042/0 (socket says 172.21.15.123:42042) 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03c3fff700 1 -- 172.21.15.123:0/2008540656 learned_addr learned my addr 172.21.15.123:0/2008540656 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03c9779700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c40b0b30 0x7f03c40b0fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.957+0000 7f03c8f78700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03b4004180 0x7f03c4149fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c3fff700 1 -- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03b4004180 msgr2=0x7f03c4149fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c3fff700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03b4004180 0x7f03c4149fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c3fff700 1 -- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c40b0b30 msgr2=0x7f03c40b0fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c3fff700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c40b0b30 0x7f03c40b0fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:01.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c3fff700 1 -- 172.21.15.123:0/2008540656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03c4155980 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c3fff700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c414b7e0 0x7f03c414bc50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03b0001e50 tx=0x7f03b0015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c0ff9700 1 -- 172.21.15.123:0/2008540656 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0001430 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c0ff9700 1 -- 172.21.15.123:0/2008540656 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b00015b0 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03c4155c60 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03c0ff9700 1 -- 172.21.15.123:0/2008540656 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0007320 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.958+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03c40b1510 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.959+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03c41563f0 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.959+0000 7f03c0ff9700 1 -- 172.21.15.123:0/2008540656 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f03b0019030 con 0x7f03c414b7e0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.960+0000 7f03c0ff9700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f03a8066f30 0x7f03a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.960+0000 7f03c8f78700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f03a8066f30 0x7f03a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:01.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.960+0000 7f03c0ff9700 1 -- 172.21.15.123:0/2008540656 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f03b0085b60 con 0x7f03c414b7e0 2023-10-06T04:32:01.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.960+0000 7f03c8f78700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f03a8066f30 0x7f03a80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f03b80010e0 tx=0x7f03b8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:01.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:01.964+0000 7f03c0ff9700 1 -- 172.21.15.123:0/2008540656 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f03b0052c70 con 0x7f03c414b7e0 2023-10-06T04:32:02.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.145+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f03c414c190 con 0x7f03c414b7e0 2023-10-06T04:32:02.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.146+0000 7f03c0ff9700 1 -- 172.21.15.123:0/2008540656 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f03b0017020 con 0x7f03c414b7e0 2023-10-06T04:32:02.149 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:02.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f03a8066f30 msgr2=0x7f03a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f03a8066f30 0x7f03a80693e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f03b80010e0 tx=0x7f03b8009040 comp rx=0 tx=0).stop 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c414b7e0 msgr2=0x7f03c414bc50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c414b7e0 0x7f03c414bc50 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f03b0001e50 tx=0x7f03b0015000 comp rx=0 tx=0).stop 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 shutdown_connections 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f03a8066f30 0x7f03a80693e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f03c40b0b30 0x7f03c40b0fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f03b4004180 0x7f03c4149fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 --2- 172.21.15.123:0/2008540656 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f03c414b7e0 0x7f03c414bc50 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 >> 172.21.15.123:0/2008540656 conn(0x7f03c40c9ca0 msgr2=0x7f03c414cec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 shutdown_connections 2023-10-06T04:32:02.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.148+0000 7f03cb1dc700 1 -- 172.21.15.123:0/2008540656 wait complete. 2023-10-06T04:32:02.162 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:32:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.280+0000 7fc6d0887700 1 Processor -- start 2023-10-06T04:32:02.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.280+0000 7fc6d0887700 1 -- start start 2023-10-06T04:32:02.284 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6d0887700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc06b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6d0887700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc6cc13fdc0 0x7fc6cc067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6d0887700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6cc06b020 con 0x7fc6cc06b9f0 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6d0887700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6cc06a450 con 0x7fc6cc13fdc0 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6d0887700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc06a6f0 con 0x7fc6cc13f9f0 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6c9d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc6cc13fdc0 0x7fc6cc067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6c9d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc6cc13fdc0 0x7fc6cc067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42048/0 (socket says 172.21.15.123:42048) 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6c9d9b700 1 -- 172.21.15.123:0/4082200517 learned_addr learned my addr 172.21.15.123:0/4082200517 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6ca59c700 1 --2- 172.21.15.123:0/4082200517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc06b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6c9d9b700 1 -- 172.21.15.123:0/4082200517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 msgr2=0x7fc6cc06b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6c9d9b700 1 --2- 172.21.15.123:0/4082200517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc06b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6c9d9b700 1 -- 172.21.15.123:0/4082200517 >> v1:172.21.15.177:6789/0 conn(0x7fc6cc06b9f0 legacy=0x7fc6cc0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.281+0000 7fc6c9d9b700 1 -- 172.21.15.123:0/4082200517 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6cc06a9d0 con 0x7fc6cc13fdc0 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6c9d9b700 1 --2- 172.21.15.123:0/4082200517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc13fdc0 0x7fc6cc067660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0099b0 tx=0x7fc6bc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6efb460c297c1d8d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6c959a700 1 -- 172.21.15.123:0/4082200517 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc009ea0 con 0x7fc6cc13fdc0 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6c959a700 1 -- 172.21.15.123:0/4082200517 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6bc00b860 con 0x7fc6cc13fdc0 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6c959a700 1 -- 172.21.15.123:0/4082200517 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6bc00bc00 con 0x7fc6cc13fdc0 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 -- 172.21.15.123:0/4082200517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc13fdc0 msgr2=0x7fc6cc067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 --2- 172.21.15.123:0/4082200517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc13fdc0 0x7fc6cc067660 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fc6bc0099b0 tx=0x7fc6bc009250 comp rx=0 tx=0).stop 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 -- 172.21.15.123:0/4082200517 shutdown_connections 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 --2- 172.21.15.123:0/4082200517 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc06b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 --2- 172.21.15.123:0/4082200517 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc13fdc0 0x7fc6cc067660 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 -- 172.21.15.123:0/4082200517 >> 172.21.15.123:0/4082200517 conn(0x7fc6cc0cd020 msgr2=0x7fc6cc0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:02.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 -- 172.21.15.123:0/4082200517 shutdown_connections 2023-10-06T04:32:02.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.282+0000 7fc6d0887700 1 -- 172.21.15.123:0/4082200517 wait complete. 2023-10-06T04:32:02.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.283+0000 7fc6d0887700 1 Processor -- start 2023-10-06T04:32:02.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.283+0000 7fc6d0887700 1 -- start start 2023-10-06T04:32:02.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6d0887700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6cc06b9f0 0x7fc6cc0c1dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6d0887700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc16a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6d0887700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc0c3440 0x7fc6cc156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6d0887700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc13b2f0 con 0x7fc6cc0c3440 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6d0887700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0c23a0 con 0x7fc6cc06b9f0 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6d0887700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6cc0c2680 con 0x7fc6cc13f9f0 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6cad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc0c3440 0x7fc6cc156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6cad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc0c3440 0x7fc6cc156570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42056/0 (socket says 172.21.15.123:42056) 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6cad9d700 1 -- 172.21.15.123:0/1242182685 learned_addr learned my addr 172.21.15.123:0/1242182685 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6ca59c700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6cc06b9f0 0x7fc6cc0c1dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.284+0000 7fc6c9d9b700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc16a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6cad9d700 1 -- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 msgr2=0x7fc6cc16a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6cad9d700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc16a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6cad9d700 1 -- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6cc06b9f0 msgr2=0x7fc6cc0c1dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6cad9d700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6cc06b9f0 0x7fc6cc0c1dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6cad9d700 1 -- 172.21.15.123:0/1242182685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6bc0094f0 con 0x7fc6cc0c3440 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6cad9d700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc0c3440 0x7fc6cc156570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c840 tx=0x7fc6c4001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6ba7fc700 1 -- 172.21.15.123:0/1242182685 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4012630 con 0x7fc6cc0c3440 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6ba7fc700 1 -- 172.21.15.123:0/1242182685 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6c40127b0 con 0x7fc6cc0c3440 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6cc0c2960 con 0x7fc6cc0c3440 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6ba7fc700 1 -- 172.21.15.123:0/1242182685 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6c4012b70 con 0x7fc6cc0c3440 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.285+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6cc0c2f20 con 0x7fc6cc0c3440 2023-10-06T04:32:02.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.286+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6cc0c3110 con 0x7fc6cc0c3440 2023-10-06T04:32:02.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.286+0000 7fc6ba7fc700 1 -- 172.21.15.123:0/1242182685 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc6c401a3a0 con 0x7fc6cc0c3440 2023-10-06T04:32:02.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.286+0000 7fc6ba7fc700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc6ac066f30 0x7fc6ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.287+0000 7fc6ba7fc700 1 -- 172.21.15.123:0/1242182685 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc6c4084890 con 0x7fc6cc0c3440 2023-10-06T04:32:02.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.287+0000 7fc6ca59c700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc6ac066f30 0x7fc6ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.287+0000 7fc6ca59c700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc6ac066f30 0x7fc6ac0693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009db0 tx=0x7fc6c00091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.291+0000 7fc6ba7fc700 1 -- 172.21.15.123:0/1242182685 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc6c4051900 con 0x7fc6cc0c3440 2023-10-06T04:32:02.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.475+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc6cc140bf0 con 0x7fc6cc0c3440 2023-10-06T04:32:02.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.475+0000 7fc6ba7fc700 1 -- 172.21.15.123:0/1242182685 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc6c4054f20 con 0x7fc6cc0c3440 2023-10-06T04:32:02.477 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:32:02.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc6ac066f30 msgr2=0x7fc6ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc6ac066f30 0x7fc6ac0693e0 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc6c0009db0 tx=0x7fc6c00091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc0c3440 msgr2=0x7fc6cc156570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc0c3440 0x7fc6cc156570 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc6c400c840 tx=0x7fc6c4001670 comp rx=0 tx=0).stop 2023-10-06T04:32:02.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 shutdown_connections 2023-10-06T04:32:02.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc6ac066f30 0x7fc6ac0693e0 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc6cc0c3440 0x7fc6cc156570 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc6cc06b9f0 0x7fc6cc0c1dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 --2- 172.21.15.123:0/1242182685 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc6cc13f9f0 0x7fc6cc16a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 >> 172.21.15.123:0/1242182685 conn(0x7fc6cc0cd020 msgr2=0x7fc6cc0b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:02.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 shutdown_connections 2023-10-06T04:32:02.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.477+0000 7fc6d0887700 1 -- 172.21.15.123:0/1242182685 wait complete. 2023-10-06T04:32:02.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:02.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.606+0000 7fcf6564e700 1 Processor -- start 2023-10-06T04:32:02.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.606+0000 7fcf6564e700 1 -- start start 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf6564e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 0x7fcf600cf170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf6564e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf600cbaf0 con 0x7fcf600b2860 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf6564e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf600b35d0 con 0x7fcf600b3130 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf6564e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf600c6830 con 0x7fcf600ce860 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5effd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcf600b2860 0x7fcf600ce150 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53114/0 (socket says 172.21.15.123:53114) 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5effd700 1 -- 172.21.15.123:0/825093937 learned_addr learned my addr 172.21.15.123:0/825093937 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5e7fc700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 0x7fcf600cf170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2732829253 0 0) 0x7fcf600cbaf0 con 0x7fcf600b2860 2023-10-06T04:32:02.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf48003570 con 0x7fcf600b2860 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1151494481 0 0) 0x7fcf48003570 con 0x7fcf600b2860 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 msgr2=0x7fcf600cf170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 0x7fcf600cf170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 >> v1:172.21.15.177:6789/0 conn(0x7fcf600b3130 legacy=0x7fcf600c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf60159aa0 con 0x7fcf600b2860 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.607+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fcf54001470 con 0x7fcf600b2860 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 >> v1:172.21.15.123:6789/0 conn(0x7fcf600b2860 legacy=0x7fcf600ce150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48003a90 0x7fcf48003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 0x7fcf48004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf48004d50 0x7fcf480051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4800bb40 con 0x7fcf48004d50 2023-10-06T04:32:02.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4800bcb0 con 0x7fcf48003a90 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf4800be70 con 0x7fcf600ce860 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5f7fe700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf48004d50 0x7fcf480051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5effd700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48003a90 0x7fcf48003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5e7fc700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 0x7fcf48004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 msgr2=0x7fcf48004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5f7fe700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600ce860 0x7fcf48004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48003a90 msgr2=0x7fcf48003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5f7fe700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf48003a90 0x7fcf48003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/825093937 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf48003570 con 0x7fcf48004d50 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5f7fe700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf48004d50 0x7fcf480051c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf58001ff0 tx=0x7fcf58001670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78c837ce5e709154 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58001980 con 0x7fcf48004d50 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800c8b0 con 0x7fcf48004d50 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.608+0000 7fcf5dffb700 1 -- 172.21.15.123:0/825093937 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf5800cc50 con 0x7fcf48004d50 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.609+0000 7fcf6564e700 1 -- 172.21.15.123:0/825093937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf48004d50 msgr2=0x7fcf480051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.609+0000 7fcf6564e700 1 --2- 172.21.15.123:0/825093937 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf48004d50 0x7fcf480051c0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7fcf58001ff0 tx=0x7fcf58001670 comp rx=0 tx=0).stop 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.609+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/825093937 reap_dead start 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.609+0000 7fcf6564e700 1 -- 172.21.15.123:0/825093937 shutdown_connections 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.609+0000 7fcf6564e700 1 -- 172.21.15.123:0/825093937 >> 172.21.15.123:0/825093937 conn(0x7fcf60063a10 msgr2=0x7fcf600d0370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:02.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.609+0000 7fcf6564e700 1 -- 172.21.15.123:0/825093937 shutdown_connections 2023-10-06T04:32:02.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.609+0000 7fcf6564e700 1 -- 172.21.15.123:0/825093937 wait complete. 2023-10-06T04:32:02.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.610+0000 7fcf6564e700 1 Processor -- start 2023-10-06T04:32:02.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.611+0000 7fcf6564e700 1 -- start start 2023-10-06T04:32:02.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.611+0000 7fcf6564e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf600b4b80 0x7fcf6016a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.611+0000 7fcf6564e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600b4f50 0x7fcf600c87d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.611+0000 7fcf6564e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6015bfa0 0x7fcf6015c440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.611+0000 7fcf6564e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf6015c980 con 0x7fcf6015bfa0 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.611+0000 7fcf6564e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf600c9070 con 0x7fcf600b4b80 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.611+0000 7fcf6564e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf600c9350 con 0x7fcf600b4f50 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6015bfa0 0x7fcf6015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6015bfa0 0x7fcf6015c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42070/0 (socket says 172.21.15.123:42070) 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/2440869077 learned_addr learned my addr 172.21.15.123:0/2440869077 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5effd700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf600b4b80 0x7fcf6016a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5e7fc700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600b4f50 0x7fcf600c87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600b4f50 msgr2=0x7fcf600c87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600b4f50 0x7fcf600c87d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf600b4b80 msgr2=0x7fcf6016a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf600b4b80 0x7fcf6016a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 -- 172.21.15.123:0/2440869077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf600c9630 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf5f7fe700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6015bfa0 0x7fcf6015c440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800bc80 tx=0x7fcf58015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf46ffd700 1 -- 172.21.15.123:0/2440869077 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58001980 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf46ffd700 1 -- 172.21.15.123:0/2440869077 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf5800c6d0 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf46ffd700 1 -- 172.21.15.123:0/2440869077 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf58006e10 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf600c9910 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.612+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf600c8d10 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.613+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf600c9e30 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.613+0000 7fcf46ffd700 1 -- 172.21.15.123:0/2440869077 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcf58019070 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.614+0000 7fcf46ffd700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.614+0000 7fcf46ffd700 1 -- 172.21.15.123:0/2440869077 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fcf58085eb0 con 0x7fcf6015bfa0 2023-10-06T04:32:02.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.614+0000 7fcf5effd700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.621 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.614+0000 7fcf5effd700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf600cf120 tx=0x7fcf54001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.621 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.618+0000 7fcf46ffd700 1 -- 172.21.15.123:0/2440869077 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcf600c9e30 con 0x7fcf6015bfa0 2023-10-06T04:32:02.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.799+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf600c9e30 con 0x7fcf6015bfa0 2023-10-06T04:32:02.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.800+0000 7fcf46ffd700 1 -- 172.21.15.123:0/2440869077 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7fcf600c9e30 con 0x7fcf6015bfa0 2023-10-06T04:32:02.802 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:02.803 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcf4c066f30 msgr2=0x7fcf4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcf4c066f30 0x7fcf4c0693e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7fcf600cf120 tx=0x7fcf54001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6015bfa0 msgr2=0x7fcf6015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6015bfa0 0x7fcf6015c440 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcf4800bc80 tx=0x7fcf58015000 comp rx=0 tx=0).stop 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 shutdown_connections 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcf4c066f30 0x7fcf4c0693e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcf6015bfa0 0x7fcf6015c440 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcf600b4b80 0x7fcf6016a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 --2- 172.21.15.123:0/2440869077 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcf600b4f50 0x7fcf600c87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 >> 172.21.15.123:0/2440869077 conn(0x7fcf60063a10 msgr2=0x7fcf600685b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:02.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 shutdown_connections 2023-10-06T04:32:02.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.802+0000 7fcf6564e700 1 -- 172.21.15.123:0/2440869077 wait complete. 2023-10-06T04:32:02.816 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:31:32.045143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:31:33.390421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:32:02.817 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:02.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.935+0000 7f23405e7700 1 Processor -- start 2023-10-06T04:32:02.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.935+0000 7f23405e7700 1 -- start start 2023-10-06T04:32:02.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.936+0000 7f23405e7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2338157430 0x7f23380ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.936+0000 7f23405e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.936+0000 7f23405e7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23380c9d70 con 0x7f23380c8f80 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.936+0000 7f23405e7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2338157800 con 0x7f2338157430 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.936+0000 7f23405e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2338159200 con 0x7f23380caf00 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233e383700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2338157430 0x7f23380ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:37452/0 (socket says 172.21.15.123:37452) 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 -- 172.21.15.123:0/471868349 learned_addr learned my addr 172.21.15.123:0/471868349 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233eb84700 1 --1- 172.21.15.123:0/471868349 >> v1:172.21.15.177:6789/0 conn(0x7f23380c8f80 0x7f233815af10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:49210/0 (socket says 172.21.15.123:49210) 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 -- 172.21.15.123:0/471868349 >> v2:172.21.15.177:3300/0 conn(0x7f2338157430 msgr2=0x7f23380ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:02.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 --2- 172.21.15.123:0/471868349 >> v2:172.21.15.177:3300/0 conn(0x7f2338157430 0x7f23380ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 -- 172.21.15.123:0/471868349 >> v1:172.21.15.177:6789/0 conn(0x7f23380c8f80 legacy=0x7f233815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 -- 172.21.15.123:0/471868349 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23380cb3a0 con 0x7f23380caf00 2023-10-06T04:32:02.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233d381700 1 -- 172.21.15.123:0/471868349 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 657411849 0 0) 0x7f2330001490 con 0x7f23380c8f80 2023-10-06T04:32:02.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233db82700 1 --2- 172.21.15.123:0/471868349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c8a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23340099b0 tx=0x7f2334009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=463f09a110277a31 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233d381700 1 -- 172.21.15.123:0/471868349 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2334009ea0 con 0x7f23380caf00 2023-10-06T04:32:02.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233d381700 1 -- 172.21.15.123:0/471868349 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f233400b860 con 0x7f23380caf00 2023-10-06T04:32:02.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.937+0000 7f233d381700 1 -- 172.21.15.123:0/471868349 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f233400bc00 con 0x7f23380caf00 2023-10-06T04:32:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 -- 172.21.15.123:0/471868349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 msgr2=0x7f23380c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 --2- 172.21.15.123:0/471868349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c8a40 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23340099b0 tx=0x7f2334009250 comp rx=0 tx=0).stop 2023-10-06T04:32:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 -- 172.21.15.123:0/471868349 shutdown_connections 2023-10-06T04:32:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 --2- 172.21.15.123:0/471868349 >> v2:172.21.15.177:3300/0 conn(0x7f2338157430 0x7f23380ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 --2- 172.21.15.123:0/471868349 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c8a40 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 -- 172.21.15.123:0/471868349 >> 172.21.15.123:0/471868349 conn(0x7f2338097450 msgr2=0x7f2338067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:02.944 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 -- 172.21.15.123:0/471868349 shutdown_connections 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.938+0000 7f23405e7700 1 -- 172.21.15.123:0/471868349 wait complete. 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.939+0000 7f23405e7700 1 Processor -- start 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.939+0000 7f23405e7700 1 -- start start 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f23405e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23380c8f80 0x7f233816b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f23405e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c5150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f23405e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2338157430 0x7f23380c5690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f23405e7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23380c6c90 con 0x7f23380c8f80 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f23405e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23381529b0 con 0x7f2338157430 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f23405e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23380ce560 con 0x7f23380caf00 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23380c8f80 0x7f233816b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23380c8f80 0x7f233816b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42086/0 (socket says 172.21.15.123:42086) 2023-10-06T04:32:02.945 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 -- 172.21.15.123:0/218521010 learned_addr learned my addr 172.21.15.123:0/218521010 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233eb84700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2338157430 0x7f23380c5690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233db82700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c5150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 -- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 msgr2=0x7f23380c5150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c5150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 -- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2338157430 msgr2=0x7f23380c5690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2338157430 0x7f23380c5690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 -- 172.21.15.123:0/218521010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23340094f0 con 0x7f23380c8f80 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f233e383700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23380c8f80 0x7f233816b1e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f232800ea30 tx=0x7f2328009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.940+0000 7f232e7fc700 1 -- 172.21.15.123:0/218521010 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800c850 con 0x7f23380c8f80 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.941+0000 7f232e7fc700 1 -- 172.21.15.123:0/218521010 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232800c9b0 con 0x7f23380c8f80 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.941+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23380c2b60 con 0x7f23380c8f80 2023-10-06T04:32:02.946 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.941+0000 7f232e7fc700 1 -- 172.21.15.123:0/218521010 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232800cd30 con 0x7f23380c8f80 2023-10-06T04:32:02.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.941+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23380c2510 con 0x7f23380c8f80 2023-10-06T04:32:02.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.942+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f233806d840 con 0x7f23380c8f80 2023-10-06T04:32:02.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.942+0000 7f232e7fc700 1 -- 172.21.15.123:0/218521010 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2328013680 con 0x7f23380c8f80 2023-10-06T04:32:02.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.942+0000 7f232e7fc700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f231c066f30 0x7f231c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:02.947 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.942+0000 7f232e7fc700 1 -- 172.21.15.123:0/218521010 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2328084c40 con 0x7f23380c8f80 2023-10-06T04:32:02.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.942+0000 7f233db82700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f231c066f30 0x7f231c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:02.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.943+0000 7f233db82700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f231c066f30 0x7f231c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f23340099b0 tx=0x7f2334017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:02.948 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:02.946+0000 7f232e7fc700 1 -- 172.21.15.123:0/218521010 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2328051d50 con 0x7f23380c8f80 2023-10-06T04:32:03.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.126+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f23380c97b0 con 0x7f23380c8f80 2023-10-06T04:32:03.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.127+0000 7f232e7fc700 1 -- 172.21.15.123:0/218521010 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f23280210a0 con 0x7f23380c8f80 2023-10-06T04:32:03.130 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:03.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f231c066f30 msgr2=0x7f231c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f231c066f30 0x7f231c0693e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f23340099b0 tx=0x7f2334017000 comp rx=0 tx=0).stop 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23380c8f80 msgr2=0x7f233816b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23380c8f80 0x7f233816b1e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f232800ea30 tx=0x7f2328009120 comp rx=0 tx=0).stop 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 shutdown_connections 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f231c066f30 0x7f231c0693e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23380c8f80 0x7f233816b1e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2338157430 0x7f23380c5690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 --2- 172.21.15.123:0/218521010 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23380caf00 0x7f23380c5150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 >> 172.21.15.123:0/218521010 conn(0x7f2338097450 msgr2=0x7f2338159690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:03.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 shutdown_connections 2023-10-06T04:32:03.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.129+0000 7f23405e7700 1 -- 172.21.15.123:0/218521010 wait complete. 2023-10-06T04:32:03.143 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:03.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.260+0000 7f64da0e7700 1 Processor -- start 2023-10-06T04:32:03.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.260+0000 7f64da0e7700 1 -- start start 2023-10-06T04:32:03.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.261+0000 7f64da0e7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f64d4154ee0 0x7f64d40c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.261+0000 7f64da0e7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d406ecb0 con 0x7f64d4067d70 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.261+0000 7f64da0e7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64d40c7f90 con 0x7f64d4158670 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.261+0000 7f64da0e7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64d406e380 con 0x7f64d4154ee0 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.261+0000 7f64d3fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f64d4067d70 0x7f64d406a3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53124/0 (socket says 172.21.15.123:53124) 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.261+0000 7f64d3fff700 1 -- 172.21.15.123:0/2981309434 learned_addr learned my addr 172.21.15.123:0/2981309434 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.261+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2981309434 >> v2:172.21.15.177:3300/0 conn(0x7f64d4154ee0 0x7f64d40c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 674478239 0 0) 0x7f64d406ecb0 con 0x7f64d4067d70 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64bc003570 con 0x7f64d4067d70 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1894556844 0 0) 0x7f64d40c7f90 con 0x7f64d4158670 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64d406ecb0 con 0x7f64d4158670 2023-10-06T04:32:03.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2040791356 0 0) 0x7f64bc003570 con 0x7f64d4067d70 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 >> v2:172.21.15.177:3300/0 conn(0x7f64d4154ee0 msgr2=0x7f64d40c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 --2- 172.21.15.123:0/2981309434 >> v2:172.21.15.177:3300/0 conn(0x7f64d4154ee0 0x7f64d40c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 >> v1:172.21.15.177:6789/0 conn(0x7f64d4158670 legacy=0x7f64d4067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d406e660 con 0x7f64d4067d70 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f64cc0017f0 con 0x7f64d4067d70 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 >> v1:172.21.15.123:6789/0 conn(0x7f64d4067d70 legacy=0x7f64d406a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64bc003bf0 0x7f64bc003fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc004810 0x7f64bc004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64bc0051c0 0x7f64bc00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d406e380 con 0x7f64bc003bf0 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc00c0b0 con 0x7f64bc004810 2023-10-06T04:32:03.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64bc00c270 con 0x7f64bc0051c0 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2981309434 >> v2:172.21.15.177:3300/0 conn(0x7f64d4154ee0 0x7f64d40c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64bc003bf0 0x7f64bc003fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d3fff700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64bc0051c0 0x7f64bc00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d2ffd700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc004810 0x7f64bc004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d37fe700 1 -- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64bc0051c0 msgr2=0x7f64bc00ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64bc0051c0 0x7f64bc00ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d37fe700 1 -- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc004810 msgr2=0x7f64bc004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc004810 0x7f64bc004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d3fff700 1 -- 172.21.15.123:0/2981309434 reap_dead start 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d37fe700 1 -- 172.21.15.123:0/2981309434 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc003570 con 0x7f64bc003bf0 2023-10-06T04:32:03.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.262+0000 7f64d2ffd700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc004810 0x7f64bc004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64bc003bf0 0x7f64bc003fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009db0 tx=0x7f64c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256c8e9f5e5d40a5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c8009740 con 0x7f64bc003bf0 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64c800e920 con 0x7f64bc003bf0 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64d27fc700 1 -- 172.21.15.123:0/2981309434 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64c800eca0 con 0x7f64bc003bf0 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64da0e7700 1 -- 172.21.15.123:0/2981309434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64bc003bf0 msgr2=0x7f64bc003fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64bc003bf0 0x7f64bc003fe0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f64c8009db0 tx=0x7f64c80091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64da0e7700 1 -- 172.21.15.123:0/2981309434 shutdown_connections 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2981309434 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64bc003bf0 0x7f64bc003fe0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64da0e7700 1 -- 172.21.15.123:0/2981309434 >> 172.21.15.123:0/2981309434 conn(0x7f64d40cd020 msgr2=0x7f64d40cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64da0e7700 1 -- 172.21.15.123:0/2981309434 shutdown_connections 2023-10-06T04:32:03.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.263+0000 7f64da0e7700 1 -- 172.21.15.123:0/2981309434 wait complete. 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.265+0000 7f64da0e7700 1 Processor -- start 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.265+0000 7f64da0e7700 1 -- start start 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64da0e7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc003bf0 0x7f64d415c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64da0e7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d4169410 0x7f64d4140a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64da0e7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40c4630 0x7f64d4156280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64da0e7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d40ccb50 con 0x7f64d40c4630 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64da0e7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64d416a410 con 0x7f64bc003bf0 2023-10-06T04:32:03.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64da0e7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64d4140fc0 con 0x7f64d4169410 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40c4630 0x7f64d4156280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40c4630 0x7f64d4156280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42106/0 (socket says 172.21.15.123:42106) 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 -- 172.21.15.123:0/2376367429 learned_addr learned my addr 172.21.15.123:0/2376367429 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d2ffd700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d4169410 0x7f64d4140a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc003bf0 0x7f64d415c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 -- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d4169410 msgr2=0x7f64d4140a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d4169410 0x7f64d4140a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 -- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc003bf0 msgr2=0x7f64d415c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc003bf0 0x7f64d415c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 -- 172.21.15.123:0/2376367429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64d4169890 con 0x7f64d40c4630 2023-10-06T04:32:03.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d2ffd700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d4169410 0x7f64d4140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc003bf0 0x7f64d415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64d3fff700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40c4630 0x7f64d4156280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64d406bb40 tx=0x7f64cc002190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64bb7fe700 1 -- 172.21.15.123:0/2376367429 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc00ace0 con 0x7f64d40c4630 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64bb7fe700 1 -- 172.21.15.123:0/2376367429 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64cc00ae60 con 0x7f64d40c4630 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64d4169b70 con 0x7f64d40c4630 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.266+0000 7f64bb7fe700 1 -- 172.21.15.123:0/2376367429 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64cc013640 con 0x7f64d40c4630 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.267+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64d416a130 con 0x7f64d40c4630 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.267+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64d406d240 con 0x7f64d40c4630 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.268+0000 7f64bb7fe700 1 -- 172.21.15.123:0/2376367429 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f64cc0139f0 con 0x7f64d40c4630 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.268+0000 7f64bb7fe700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64c0066f30 0x7f64c00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.268+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64c0066f30 0x7f64c00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.268+0000 7f64bb7fe700 1 -- 172.21.15.123:0/2376367429 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f64cc085120 con 0x7f64d40c4630 2023-10-06T04:32:03.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.269+0000 7f64d37fe700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64c0066f30 0x7f64c00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ca160 tx=0x7f64c8019000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.276 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.273+0000 7f64bb7fe700 1 -- 172.21.15.123:0/2376367429 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f64cc052190 con 0x7f64d40c4630 2023-10-06T04:32:03.401 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:03.403 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.458+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f64d404bb80 con 0x7f64d40c4630 2023-10-06T04:32:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.458+0000 7f64bb7fe700 1 -- 172.21.15.123:0/2376367429 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f64cc0557b0 con 0x7f64d40c4630 2023-10-06T04:32:03.461 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64c0066f30 msgr2=0x7f64c00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64c0066f30 0x7f64c00693e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f64d40ca160 tx=0x7f64c8019000 comp rx=0 tx=0).stop 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40c4630 msgr2=0x7f64d4156280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40c4630 0x7f64d4156280 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f64d406bb40 tx=0x7f64cc002190 comp rx=0 tx=0).stop 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 shutdown_connections 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f64c0066f30 0x7f64c00693e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f64d40c4630 0x7f64d4156280 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f64bc003bf0 0x7f64d415c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 --2- 172.21.15.123:0/2376367429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f64d4169410 0x7f64d4140a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 >> 172.21.15.123:0/2376367429 conn(0x7f64d40cd020 msgr2=0x7f64d415a0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:03.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 shutdown_connections 2023-10-06T04:32:03.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.460+0000 7f64da0e7700 1 -- 172.21.15.123:0/2376367429 wait complete. 2023-10-06T04:32:03.474 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:32:03.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.592+0000 7f1a532d6700 1 Processor -- start 2023-10-06T04:32:03.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.592+0000 7f1a532d6700 1 -- start start 2023-10-06T04:32:03.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.593+0000 7f1a532d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c13fa70 0x7f1a4c0c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.593+0000 7f1a532d6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4c06a4e0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.593+0000 7f1a532d6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a4c0c8ec0 con 0x7f1a4c0cabf0 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.593+0000 7f1a532d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c91a0 con 0x7f1a4c13fa70 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.593+0000 7f1a51873700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1a4c06e9f0 0x7f1a4c159e10 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53138/0 (socket says 172.21.15.123:53138) 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.593+0000 7f1a51873700 1 -- 172.21.15.123:0/893904471 learned_addr learned my addr 172.21.15.123:0/893904471 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.593+0000 7f1a51072700 1 --2- 172.21.15.123:0/893904471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c13fa70 0x7f1a4c0c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1417328086 0 0) 0x7f1a4c06a4e0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a34003570 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4124433221 0 0) 0x7f1a4c0c8ec0 con 0x7f1a4c0cabf0 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a4c06a4e0 con 0x7f1a4c0cabf0 2023-10-06T04:32:03.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4272150439 0 0) 0x7f1a34003570 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a51072700 1 -- 172.21.15.123:0/893904471 >> v1:172.21.15.177:6789/0 conn(0x7f1a4c0cabf0 legacy=0x7f1a4c06e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a51072700 1 -- 172.21.15.123:0/893904471 >> v1:172.21.15.123:6789/0 conn(0x7f1a4c06e9f0 legacy=0x7f1a4c159e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a51072700 1 -- 172.21.15.123:0/893904471 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c0c9480 con 0x7f1a4c13fa70 2023-10-06T04:32:03.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a51072700 1 --2- 172.21.15.123:0/893904471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c13fa70 0x7f1a4c0c8300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009e90 tx=0x7f1a3c009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd59e9be7e9d801b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c00e070 con 0x7f1a4c13fa70 2023-10-06T04:32:03.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c00b870 con 0x7f1a4c13fa70 2023-10-06T04:32:03.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.594+0000 7f1a4bfff700 1 -- 172.21.15.123:0/893904471 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a44001470 con 0x7f1a4c13fa70 2023-10-06T04:32:03.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.595+0000 7f1a532d6700 1 -- 172.21.15.123:0/893904471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c13fa70 msgr2=0x7f1a4c0c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.595+0000 7f1a532d6700 1 --2- 172.21.15.123:0/893904471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c13fa70 0x7f1a4c0c8300 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f1a3c009e90 tx=0x7f1a3c009780 comp rx=0 tx=0).stop 2023-10-06T04:32:03.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.595+0000 7f1a532d6700 1 -- 172.21.15.123:0/893904471 shutdown_connections 2023-10-06T04:32:03.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.595+0000 7f1a532d6700 1 --2- 172.21.15.123:0/893904471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c13fa70 0x7f1a4c0c8300 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.595+0000 7f1a532d6700 1 -- 172.21.15.123:0/893904471 >> 172.21.15.123:0/893904471 conn(0x7f1a4c0cd020 msgr2=0x7f1a4c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:03.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.595+0000 7f1a532d6700 1 -- 172.21.15.123:0/893904471 shutdown_connections 2023-10-06T04:32:03.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.595+0000 7f1a532d6700 1 -- 172.21.15.123:0/893904471 wait complete. 2023-10-06T04:32:03.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.596+0000 7f1a532d6700 1 Processor -- start 2023-10-06T04:32:03.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.597+0000 7f1a532d6700 1 -- start start 2023-10-06T04:32:03.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.597+0000 7f1a532d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a4c06e9f0 0x7f1a4c0c6b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a532d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c0cabf0 0x7f1a4c0c3380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a532d6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4c13fa70 0x7f1a4c152f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a532d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c6710 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a532d6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c4ea0 con 0x7f1a4c13fa70 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a532d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a4c0c70c0 con 0x7f1a4c0cabf0 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a4c06e9f0 0x7f1a4c0c6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a4c06e9f0 0x7f1a4c0c6b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42120/0 (socket says 172.21.15.123:42120) 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 -- 172.21.15.123:0/723583333 learned_addr learned my addr 172.21.15.123:0/723583333 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51873700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4c13fa70 0x7f1a4c152f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a50871700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c0cabf0 0x7f1a4c0c3380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 -- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c0cabf0 msgr2=0x7f1a4c0c3380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c0cabf0 0x7f1a4c0c3380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 -- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4c13fa70 msgr2=0x7f1a4c152f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4c13fa70 0x7f1a4c152f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 -- 172.21.15.123:0/723583333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a3c009a20 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a51072700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a4c06e9f0 0x7f1a4c0c6b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c063d40 tx=0x7f1a3c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a497fa700 1 -- 172.21.15.123:0/723583333 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c021070 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a497fa700 1 -- 172.21.15.123:0/723583333 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a3c01c2e0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a497fa700 1 -- 172.21.15.123:0/723583333 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a3c01cb90 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a4c12e410 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.598+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c0ad4a0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.599+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1a4c06c210 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.599+0000 7f1a497fa700 1 -- 172.21.15.123:0/723583333 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a3c01a4e0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.600+0000 7f1a497fa700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a38066f30 0x7f1a380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.600+0000 7f1a497fa700 1 -- 172.21.15.123:0/723583333 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a3c0875a0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.600+0000 7f1a50871700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a38066f30 0x7f1a380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.600+0000 7f1a50871700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a38066f30 0x7f1a380693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c15aeb0 tx=0x7f1a40009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.604+0000 7f1a497fa700 1 -- 172.21.15.123:0/723583333 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1a3c0546b0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.784+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1a4c04bb80 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.784+0000 7f1a497fa700 1 -- 172.21.15.123:0/723583333 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1a3c057cd0 con 0x7f1a4c06e9f0 2023-10-06T04:32:03.787 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:03.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.786+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a38066f30 msgr2=0x7f1a380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.786+0000 7f1a532d6700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a38066f30 0x7f1a380693e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c15aeb0 tx=0x7f1a40009040 comp rx=0 tx=0).stop 2023-10-06T04:32:03.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.786+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a4c06e9f0 msgr2=0x7f1a4c0c6b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.786+0000 7f1a532d6700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a4c06e9f0 0x7f1a4c0c6b80 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c063d40 tx=0x7f1a3c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:03.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 shutdown_connections 2023-10-06T04:32:03.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a38066f30 0x7f1a380693e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a4c06e9f0 0x7f1a4c0c6b80 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a4c13fa70 0x7f1a4c152f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 --2- 172.21.15.123:0/723583333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a4c0cabf0 0x7f1a4c0c3380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 >> 172.21.15.123:0/723583333 conn(0x7f1a4c0cd020 msgr2=0x7f1a4c13ca90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:03.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 shutdown_connections 2023-10-06T04:32:03.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.787+0000 7f1a532d6700 1 -- 172.21.15.123:0/723583333 wait complete. 2023-10-06T04:32:03.801 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:32:03.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.919+0000 7f3a7b07e700 1 Processor -- start 2023-10-06T04:32:03.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.919+0000 7f3a7b07e700 1 -- start start 2023-10-06T04:32:03.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a7b07e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 0x7f3a740c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a7b07e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3a74068670 0x7f3a74068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a7b07e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3a74069aa0 con 0x7f3a740c8340 2023-10-06T04:32:03.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a7b07e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3a740aa500 con 0x7f3a74068670 2023-10-06T04:32:03.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a7b07e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74069050 con 0x7f3a740c8710 2023-10-06T04:32:03.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a78e1a700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3a740c8340 0x7f3a740a9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53152/0 (socket says 172.21.15.123:53152) 2023-10-06T04:32:03.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a78e1a700 1 -- 172.21.15.123:0/48760053 learned_addr learned my addr 172.21.15.123:0/48760053 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a73fff700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 0x7f3a740c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a7961b700 1 --2- 172.21.15.123:0/48760053 >> v2:172.21.15.177:3300/0 conn(0x7f3a74068670 0x7f3a74068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 930976876 0 0) 0x7f3a74069aa0 con 0x7f3a740c8340 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.920+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3a64003570 con 0x7f3a740c8340 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1309678640 0 0) 0x7f3a64003570 con 0x7f3a740c8340 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 msgr2=0x7f3a740c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 0x7f3a740c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 >> v2:172.21.15.177:3300/0 conn(0x7f3a74068670 msgr2=0x7f3a74068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 --2- 172.21.15.123:0/48760053 >> v2:172.21.15.177:3300/0 conn(0x7f3a74068670 0x7f3a74068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a7406a340 con 0x7f3a740c8340 2023-10-06T04:32:03.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f3a68001470 con 0x7f3a740c8340 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a7961b700 1 --2- 172.21.15.123:0/48760053 >> v2:172.21.15.177:3300/0 conn(0x7f3a74068670 0x7f3a74068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 >> v1:172.21.15.123:6789/0 conn(0x7f3a740c8340 legacy=0x7f3a740a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a64003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 0x7f3a64004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a64004d50 0x7f3a640051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a74069050 con 0x7f3a64003a90 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6400ba60 con 0x7f3a64004d50 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a6400bba0 con 0x7f3a740c8710 2023-10-06T04:32:03.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a78e1a700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a64003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a73fff700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 0x7f3a64004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a7961b700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a64004d50 0x7f3a640051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a78e1a700 1 -- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 msgr2=0x7f3a64004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a78e1a700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740c8710 0x7f3a64004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a78e1a700 1 -- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a64004d50 msgr2=0x7f3a640051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a78e1a700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a64004d50 0x7f3a640051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a78e1a700 1 -- 172.21.15.123:0/48760053 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a64003570 con 0x7f3a64003a90 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a78e1a700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a64003ea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c3c0 tx=0x7f3a68001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8031eac4199277b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a6800cc30 con 0x7f3a64003a90 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.921+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a6800cd90 con 0x7f3a64003a90 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a737fe700 1 -- 172.21.15.123:0/48760053 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68005b50 con 0x7f3a64003a90 2023-10-06T04:32:03.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a7b07e700 1 -- 172.21.15.123:0/48760053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 msgr2=0x7f3a64003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a7b07e700 1 --2- 172.21.15.123:0/48760053 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a64003ea0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f3a6800c3c0 tx=0x7f3a68001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a7961b700 1 -- 172.21.15.123:0/48760053 reap_dead start 2023-10-06T04:32:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a7b07e700 1 -- 172.21.15.123:0/48760053 shutdown_connections 2023-10-06T04:32:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a7b07e700 1 -- 172.21.15.123:0/48760053 >> 172.21.15.123:0/48760053 conn(0x7f3a740c9ca0 msgr2=0x7f3a7406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a7b07e700 1 -- 172.21.15.123:0/48760053 shutdown_connections 2023-10-06T04:32:03.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.922+0000 7f3a7b07e700 1 -- 172.21.15.123:0/48760053 wait complete. 2023-10-06T04:32:03.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.923+0000 7f3a7b07e700 1 Processor -- start 2023-10-06T04:32:03.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.923+0000 7f3a7b07e700 1 -- start start 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a7b07e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a74157cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a7b07e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740b12d0 0x7f3a740b1720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a7b07e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a740c60c0 0x7f3a740c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a7b07e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740c79a0 con 0x7f3a64003a90 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a7b07e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3a740b1c60 con 0x7f3a740c60c0 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a7b07e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3a7415c000 con 0x7f3a740b12d0 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a78e1a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a74157cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a78e1a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a74157cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42136/0 (socket says 172.21.15.123:42136) 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a78e1a700 1 -- 172.21.15.123:0/1625353071 learned_addr learned my addr 172.21.15.123:0/1625353071 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a7961b700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a740c60c0 0x7f3a740c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.924+0000 7f3a73fff700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740b12d0 0x7f3a740b1720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a78e1a700 1 -- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740b12d0 msgr2=0x7f3a740b1720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a78e1a700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740b12d0 0x7f3a740b1720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a78e1a700 1 -- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a740c60c0 msgr2=0x7f3a740c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a78e1a700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a740c60c0 0x7f3a740c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a78e1a700 1 -- 172.21.15.123:0/1625353071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3a740c6e20 con 0x7f3a64003a90 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a78e1a700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a74157cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a68001e90 tx=0x7f3a68015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a70ff9700 1 -- 172.21.15.123:0/1625353071 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a680072e0 con 0x7f3a64003a90 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a70ff9700 1 -- 172.21.15.123:0/1625353071 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3a680074a0 con 0x7f3a64003a90 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3a740c7100 con 0x7f3a64003a90 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a70ff9700 1 -- 172.21.15.123:0/1625353071 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3a68007860 con 0x7f3a64003a90 2023-10-06T04:32:03.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.925+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3a740c76c0 con 0x7f3a64003a90 2023-10-06T04:32:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.926+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3a7415c200 con 0x7f3a64003a90 2023-10-06T04:32:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.926+0000 7f3a70ff9700 1 -- 172.21.15.123:0/1625353071 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3a6801e640 con 0x7f3a64003a90 2023-10-06T04:32:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.926+0000 7f3a70ff9700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3a58066f60 0x7f3a58069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.927+0000 7f3a70ff9700 1 -- 172.21.15.123:0/1625353071 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f3a68085ee0 con 0x7f3a64003a90 2023-10-06T04:32:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.927+0000 7f3a73fff700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3a58066f60 0x7f3a58069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.927+0000 7f3a73fff700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3a58066f60 0x7f3a58069410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3a60006f00 tx=0x7f3a60008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:03.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:03.931+0000 7f3a70ff9700 1 -- 172.21.15.123:0/1625353071 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3a7415c200 con 0x7f3a64003a90 2023-10-06T04:32:04.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.112+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f3a7415c1c0 con 0x7f3a64003a90 2023-10-06T04:32:04.115 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.112+0000 7f3a70ff9700 1 -- 172.21.15.123:0/1625353071 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f3a68052f50 con 0x7f3a64003a90 2023-10-06T04:32:04.115 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:32:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3a58066f60 msgr2=0x7f3a58069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3a58066f60 0x7f3a58069410 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f3a60006f00 tx=0x7f3a60008040 comp rx=0 tx=0).stop 2023-10-06T04:32:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 msgr2=0x7f3a74157cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a74157cf0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f3a68001e90 tx=0x7f3a68015000 comp rx=0 tx=0).stop 2023-10-06T04:32:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 shutdown_connections 2023-10-06T04:32:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3a58066f60 0x7f3a58069410 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3a64003a90 0x7f3a74157cf0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3a740c60c0 0x7f3a740c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 --2- 172.21.15.123:0/1625353071 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3a740b12d0 0x7f3a740b1720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.114+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 >> 172.21.15.123:0/1625353071 conn(0x7f3a740c9ca0 msgr2=0x7f3a741588f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:04.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.115+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 shutdown_connections 2023-10-06T04:32:04.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.115+0000 7f3a7b07e700 1 -- 172.21.15.123:0/1625353071 wait complete. 2023-10-06T04:32:04.129 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:04.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.245+0000 7f7edd613700 1 Processor -- start 2023-10-06T04:32:04.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.245+0000 7f7edd613700 1 -- start start 2023-10-06T04:32:04.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.245+0000 7f7edd613700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 0x7f7ed80c98f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7edd613700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7ed806e2c0 0x7f7ed806e740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7edd613700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ed815d1f0 con 0x7f7ed8159a30 2023-10-06T04:32:04.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7edd613700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ed806ec80 con 0x7f7ed806e2c0 2023-10-06T04:32:04.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7edd613700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8159ea0 con 0x7f7ed8156ea0 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed67fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7ed8159a30 0x7f7ed815d680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53164/0 (socket says 172.21.15.123:53164) 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed67fc700 1 -- 172.21.15.123:0/984366012 learned_addr learned my addr 172.21.15.123:0/984366012 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 0x7f7ed80c98f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed77fe700 1 --2- 172.21.15.123:0/984366012 >> v2:172.21.15.177:3300/0 conn(0x7f7ed806e2c0 0x7f7ed806e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952674515 0 0) 0x7f7ed815d1f0 con 0x7f7ed8159a30 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7ec0003570 con 0x7f7ed8159a30 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1764157103 0 0) 0x7f7ec0003570 con 0x7f7ed8159a30 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 msgr2=0x7f7ed80c98f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 0x7f7ed80c98f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 >> v2:172.21.15.177:3300/0 conn(0x7f7ed806e2c0 msgr2=0x7f7ed806e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 --2- 172.21.15.123:0/984366012 >> v2:172.21.15.177:3300/0 conn(0x7f7ed806e2c0 0x7f7ed806e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed815a130 con 0x7f7ed8159a30 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f7ecc001470 con 0x7f7ed8159a30 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 >> v1:172.21.15.123:6789/0 conn(0x7f7ed8159a30 legacy=0x7f7ed815d680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec0003a60 0x7f7ec0003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 0x7f7ec0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0004d50 0x7f7ec00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed806ec80 con 0x7f7ec0003a60 2023-10-06T04:32:04.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed8159ea0 con 0x7f7ec0004d50 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ec000ba60 con 0x7f7ed8156ea0 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec0003a60 0x7f7ec0003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed67fc700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 0x7f7ec0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed77fe700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0004d50 0x7f7ec00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.246+0000 7f7ed6ffd700 1 -- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 msgr2=0x7f7ec0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed8156ea0 0x7f7ec0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed6ffd700 1 -- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0004d50 msgr2=0x7f7ec00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0004d50 0x7f7ec00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed6ffd700 1 -- 172.21.15.123:0/984366012 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ec0003570 con 0x7f7ec0003a60 2023-10-06T04:32:04.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec0003a60 0x7f7ec0003e70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009c00 tx=0x7f7ec80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=388aa293c9457814 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800e6b0 con 0x7f7ec0003a60 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ec800e810 con 0x7f7ec0003a60 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed5ffb700 1 -- 172.21.15.123:0/984366012 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ec800ebd0 con 0x7f7ec0003a60 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7edd613700 1 -- 172.21.15.123:0/984366012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec0003a60 msgr2=0x7f7ec0003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7edd613700 1 --2- 172.21.15.123:0/984366012 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ec0003a60 0x7f7ec0003e70 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009c00 tx=0x7f7ec80091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7ed77fe700 1 -- 172.21.15.123:0/984366012 reap_dead start 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7edd613700 1 -- 172.21.15.123:0/984366012 shutdown_connections 2023-10-06T04:32:04.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7edd613700 1 -- 172.21.15.123:0/984366012 >> 172.21.15.123:0/984366012 conn(0x7f7ed8068770 msgr2=0x7f7ed8068b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:04.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7edd613700 1 -- 172.21.15.123:0/984366012 shutdown_connections 2023-10-06T04:32:04.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.247+0000 7f7edd613700 1 -- 172.21.15.123:0/984366012 wait complete. 2023-10-06T04:32:04.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 Processor -- start 2023-10-06T04:32:04.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 -- start start 2023-10-06T04:32:04.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0003a60 0x7f7ed8153ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ed80b1730 0x7f7ed8169fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed80b1ba0 0x7f7ed80c1330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed80afe80 con 0x7f7ed80b1730 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed816aa80 con 0x7f7ec0003a60 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7edd613700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ed816ad60 con 0x7f7ed80b1ba0 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7ed67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ed80b1730 0x7f7ed8169fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7ed67fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ed80b1730 0x7f7ed8169fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42168/0 (socket says 172.21.15.123:42168) 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.249+0000 7f7ed67fc700 1 -- 172.21.15.123:0/2312035514 learned_addr learned my addr 172.21.15.123:0/2312035514 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0003a60 0x7f7ed8153ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed77fe700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed80b1ba0 0x7f7ed80c1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed67fc700 1 -- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed80b1ba0 msgr2=0x7f7ed80c1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed67fc700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed80b1ba0 0x7f7ed80c1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed67fc700 1 -- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0003a60 msgr2=0x7f7ed8153ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed67fc700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0003a60 0x7f7ed8153ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed67fc700 1 -- 172.21.15.123:0/2312035514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ed80b00d0 con 0x7f7ed80b1730 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ed67fc700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ed80b1730 0x7f7ed8169fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00eaf0 tx=0x7f7ecc000dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ebeffd700 1 -- 172.21.15.123:0/2312035514 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00ee00 con 0x7f7ed80b1730 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ebeffd700 1 -- 172.21.15.123:0/2312035514 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc00c7e0 con 0x7f7ed80b1730 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7ebeffd700 1 -- 172.21.15.123:0/2312035514 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00cba0 con 0x7f7ed80b1730 2023-10-06T04:32:04.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ed80b03b0 con 0x7f7ed80b1730 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.250+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ed816a500 con 0x7f7ed80b1730 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.251+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ed816af20 con 0x7f7ed80b1730 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.251+0000 7f7ebeffd700 1 -- 172.21.15.123:0/2312035514 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7ecc01d460 con 0x7f7ed80b1730 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.252+0000 7f7ebeffd700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7ec4066f30 0x7f7ec40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.252+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7ec4066f30 0x7f7ec40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.252+0000 7f7ebeffd700 1 -- 172.21.15.123:0/2312035514 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7ecc084610 con 0x7f7ed80b1730 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.252+0000 7f7ed6ffd700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7ec4066f30 0x7f7ec40693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80c0570 tx=0x7f7ec801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.256+0000 7f7ebeffd700 1 -- 172.21.15.123:0/2312035514 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7ecc051720 con 0x7f7ed80b1730 2023-10-06T04:32:04.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.440+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f7ed8064e20 con 0x7f7ed80b1730 2023-10-06T04:32:04.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.441+0000 7f7ebeffd700 1 -- 172.21.15.123:0/2312035514 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f7ecc054d40 con 0x7f7ed80b1730 2023-10-06T04:32:04.444 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:04.444 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:04.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7ec4066f30 msgr2=0x7f7ec40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7ec4066f30 0x7f7ec40693e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7ed80c0570 tx=0x7f7ec801a000 comp rx=0 tx=0).stop 2023-10-06T04:32:04.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ed80b1730 msgr2=0x7f7ed8169fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ed80b1730 0x7f7ed8169fc0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00eaf0 tx=0x7f7ecc000dd0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 shutdown_connections 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7ec4066f30 0x7f7ec40693e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7ed80b1ba0 0x7f7ed80c1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7ec0003a60 0x7f7ed8153ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 --2- 172.21.15.123:0/2312035514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7ed80b1730 0x7f7ed8169fc0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 >> 172.21.15.123:0/2312035514 conn(0x7f7ed8068770 msgr2=0x7f7ed80cab80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 shutdown_connections 2023-10-06T04:32:04.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.443+0000 7f7edd613700 1 -- 172.21.15.123:0/2312035514 wait complete. 2023-10-06T04:32:04.458 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:31:32.045143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:31:33.390421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:32:04.458 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.575+0000 7f4c0272d700 1 Processor -- start 2023-10-06T04:32:04.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.575+0000 7f4c0272d700 1 -- start start 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.576+0000 7f4c0272d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4bfc15a1e0 0x7f4bfc06a860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.576+0000 7f4c0272d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc06af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.576+0000 7f4c0272d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bfc06b9c0 con 0x7f4bfc06b4d0 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.576+0000 7f4c0272d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bfc069aa0 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.576+0000 7f4c0272d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc069d80 con 0x7f4bfc15a5b0 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc06af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfbfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4bfc15a1e0 0x7f4bfc06a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc06af90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:37540/0 (socket says 172.21.15.123:37540) 2023-10-06T04:32:04.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/604458445 learned_addr learned my addr 172.21.15.123:0/604458445 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfaffd700 1 -- 172.21.15.123:0/604458445 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1148457405 0 0) 0x7f4bfc06b9c0 con 0x7f4bfc06b4d0 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfaffd700 1 -- 172.21.15.123:0/604458445 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4be4003570 con 0x7f4bfc06b4d0 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/604458445 >> v2:172.21.15.177:3300/0 conn(0x7f4bfc15a1e0 msgr2=0x7f4bfc06a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 --2- 172.21.15.123:0/604458445 >> v2:172.21.15.177:3300/0 conn(0x7f4bfc15a1e0 0x7f4bfc06a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/604458445 >> v1:172.21.15.177:6789/0 conn(0x7f4bfc06b4d0 legacy=0x7f4bfc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/604458445 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bfc06a060 con 0x7f4bfc15a5b0 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfbfff700 1 --2- 172.21.15.123:0/604458445 >> v2:172.21.15.177:3300/0 conn(0x7f4bfc15a1e0 0x7f4bfc06a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfb7fe700 1 --2- 172.21.15.123:0/604458445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc06af90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0099b0 tx=0x7f4bec009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2e5a3898110ad6c8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.577+0000 7f4bfaffd700 1 -- 172.21.15.123:0/604458445 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00b680 con 0x7f4bfc15a5b0 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4bfaffd700 1 -- 172.21.15.123:0/604458445 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec00b7e0 con 0x7f4bfc15a5b0 2023-10-06T04:32:04.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4bfaffd700 1 -- 172.21.15.123:0/604458445 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00bb80 con 0x7f4bfc15a5b0 2023-10-06T04:32:04.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 -- 172.21.15.123:0/604458445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 msgr2=0x7f4bfc06af90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 --2- 172.21.15.123:0/604458445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc06af90 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f4bec0099b0 tx=0x7f4bec009250 comp rx=0 tx=0).stop 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 -- 172.21.15.123:0/604458445 shutdown_connections 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 --2- 172.21.15.123:0/604458445 >> v2:172.21.15.177:3300/0 conn(0x7f4bfc15a1e0 0x7f4bfc06a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 --2- 172.21.15.123:0/604458445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc06af90 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 -- 172.21.15.123:0/604458445 >> 172.21.15.123:0/604458445 conn(0x7f4bfc0c9ca0 msgr2=0x7f4bfc06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 -- 172.21.15.123:0/604458445 shutdown_connections 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.578+0000 7f4c0272d700 1 -- 172.21.15.123:0/604458445 wait complete. 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.579+0000 7f4c0272d700 1 Processor -- start 2023-10-06T04:32:04.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.579+0000 7f4c0272d700 1 -- start start 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4c0272d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bfc06b4d0 0x7f4bfc169990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4c0272d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bfc15a1e0 0x7f4bfc0c2370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4c0272d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc0c14c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4c0272d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc154b80 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4c0272d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc154e20 con 0x7f4bfc06b4d0 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4c0272d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bfc155100 con 0x7f4bfc15a5b0 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4bfb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bfc15a1e0 0x7f4bfc0c2370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4bfb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bfc15a1e0 0x7f4bfc0c2370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42178/0 (socket says 172.21.15.123:42178) 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/568792374 learned_addr learned my addr 172.21.15.123:0/568792374 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.580+0000 7f4bfbfff700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bfc06b4d0 0x7f4bfc169990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4c00cca700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc0c14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 msgr2=0x7f4bfc0c14c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4bfb7fe700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc0c14c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bfc06b4d0 msgr2=0x7f4bfc169990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4bfb7fe700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bfc06b4d0 0x7f4bfc169990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4bfb7fe700 1 -- 172.21.15.123:0/568792374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0009420 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4bfbfff700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bfc06b4d0 0x7f4bfc169990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4bfb7fe700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bfc15a1e0 0x7f4bfc0c2370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009290 tx=0x7f4bec017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4be3fff700 1 -- 172.21.15.123:0/568792374 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec00bb60 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4be3fff700 1 -- 172.21.15.123:0/568792374 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec01a450 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bec0094f0 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4be3fff700 1 -- 172.21.15.123:0/568792374 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec01a810 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.581+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bfc1556c0 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.582+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bfc1558e0 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.583+0000 7f4be3fff700 1 -- 172.21.15.123:0/568792374 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bec01ab80 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.583+0000 7f4be3fff700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f30 0x7f4be80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.583+0000 7f4be3fff700 1 -- 172.21.15.123:0/568792374 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4bec086090 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.583+0000 7f4bfbfff700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f30 0x7f4be80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.583+0000 7f4bfbfff700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f30 0x7f4be80693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00093f0 tx=0x7f4bf0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.587+0000 7f4be3fff700 1 -- 172.21.15.123:0/568792374 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bec0531a0 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.766+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4bfc06a340 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.766+0000 7f4be3fff700 1 -- 172.21.15.123:0/568792374 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4bec0567c0 con 0x7f4bfc15a1e0 2023-10-06T04:32:04.769 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:04.770 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f30 msgr2=0x7f4be80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f30 0x7f4be80693e0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4bf00093f0 tx=0x7f4bf0008040 comp rx=0 tx=0).stop 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bfc15a1e0 msgr2=0x7f4bfc0c2370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bfc15a1e0 0x7f4bfc0c2370 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f4bec009290 tx=0x7f4bec017000 comp rx=0 tx=0).stop 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 shutdown_connections 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f30 0x7f4be80693e0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4bfc15a5b0 0x7f4bfc0c14c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4bfc06b4d0 0x7f4bfc169990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 --2- 172.21.15.123:0/568792374 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4bfc15a1e0 0x7f4bfc0c2370 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 >> 172.21.15.123:0/568792374 conn(0x7f4bfc0c9ca0 msgr2=0x7f4bfc0ad7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 shutdown_connections 2023-10-06T04:32:04.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.768+0000 7f4c0272d700 1 -- 172.21.15.123:0/568792374 wait complete. 2023-10-06T04:32:04.783 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:04.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.901+0000 7f409a373700 1 Processor -- start 2023-10-06T04:32:04.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.901+0000 7f409a373700 1 -- start start 2023-10-06T04:32:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f409a373700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 0x7f40940abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f409a373700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940c7810 con 0x7f4094068d80 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f409a373700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40940ac9f0 con 0x7f40940ac470 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f409a373700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40940c6ee0 con 0x7f4094157080 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4098910700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4094068d80 0x7f40940c7ca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53168/0 (socket says 172.21.15.123:53168) 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4098910700 1 -- 172.21.15.123:0/3885567556 learned_addr learned my addr 172.21.15.123:0/3885567556 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4093fff700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 0x7f40940abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1889634751 0 0) 0x7f40940c7810 con 0x7f4094068d80 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4080003570 con 0x7f4094068d80 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 315434640 0 0) 0x7f4080003570 con 0x7f4094068d80 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 msgr2=0x7f40940abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 0x7f40940abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 >> v1:172.21.15.177:6789/0 conn(0x7f40940ac470 legacy=0x7f4094068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40940c71c0 con 0x7f4094068d80 2023-10-06T04:32:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f408c0017f0 con 0x7f4094068d80 2023-10-06T04:32:04.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.902+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 >> v1:172.21.15.123:6789/0 conn(0x7f4094068d80 legacy=0x7f40940c7ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4080003a90 0x7f4080003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 0x7f4080004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4080004d50 0x7f40800051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40940c6ee0 con 0x7f4080003a90 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408000bb10 con 0x7f4080004d50 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f408000bcd0 con 0x7f4094157080 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4093fff700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4080003a90 0x7f4080003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f40937fe700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 0x7f4080004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4098910700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4080004d50 0x7f40800051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4093fff700 1 -- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 msgr2=0x7f4080004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4093fff700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4094157080 0x7f4080004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4093fff700 1 -- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4080004d50 msgr2=0x7f40800051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4093fff700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4080004d50 0x7f40800051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4093fff700 1 -- 172.21.15.123:0/3885567556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4080003570 con 0x7f4080003a90 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4093fff700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4080003a90 0x7f4080003e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f407c009c00 tx=0x7f407c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c592ffff5599762 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00e710 con 0x7f4080003a90 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c00e870 con 0x7f4080003a90 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.903+0000 7f4092ffd700 1 -- 172.21.15.123:0/3885567556 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c00ec30 con 0x7f4080003a90 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.904+0000 7f409a373700 1 -- 172.21.15.123:0/3885567556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4080003a90 msgr2=0x7f4080003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.904+0000 7f409a373700 1 --2- 172.21.15.123:0/3885567556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4080003a90 0x7f4080003e60 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f407c009c00 tx=0x7f407c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.904+0000 7f4098910700 1 -- 172.21.15.123:0/3885567556 reap_dead start 2023-10-06T04:32:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.904+0000 7f409a373700 1 -- 172.21.15.123:0/3885567556 shutdown_connections 2023-10-06T04:32:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.904+0000 7f409a373700 1 -- 172.21.15.123:0/3885567556 >> 172.21.15.123:0/3885567556 conn(0x7f40940c9ca0 msgr2=0x7f409406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.904+0000 7f409a373700 1 -- 172.21.15.123:0/3885567556 shutdown_connections 2023-10-06T04:32:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.904+0000 7f409a373700 1 -- 172.21.15.123:0/3885567556 wait complete. 2023-10-06T04:32:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.905+0000 7f409a373700 1 Processor -- start 2023-10-06T04:32:04.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.905+0000 7f409a373700 1 -- start start 2023-10-06T04:32:04.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f409a373700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4080003a90 0x7f409416aee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f409a373700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4094159d20 0x7f40941581c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f409a373700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4094152da0 0x7f4094067ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f409a373700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40940c0330 con 0x7f4094159d20 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f409a373700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4094158700 con 0x7f4094152da0 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f409a373700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f409415a190 con 0x7f4080003a90 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f40937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4094159d20 0x7f40941581c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f40937fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4094159d20 0x7f40941581c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42196/0 (socket says 172.21.15.123:42196) 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.906+0000 7f40937fe700 1 -- 172.21.15.123:0/1038251017 learned_addr learned my addr 172.21.15.123:0/1038251017 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f4093fff700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4080003a90 0x7f409416aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f4098910700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4094152da0 0x7f4094067ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f40937fe700 1 -- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4080003a90 msgr2=0x7f409416aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f40937fe700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4080003a90 0x7f409416aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f40937fe700 1 -- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4094152da0 msgr2=0x7f4094067ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f40937fe700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4094152da0 0x7f4094067ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f40937fe700 1 -- 172.21.15.123:0/1038251017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f409416b4c0 con 0x7f4094159d20 2023-10-06T04:32:04.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f40937fe700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4094159d20 0x7f40941581c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4084002b00 tx=0x7f4084007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f408bfff700 1 -- 172.21.15.123:0/1038251017 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408400f8d0 con 0x7f4094159d20 2023-10-06T04:32:04.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f408bfff700 1 -- 172.21.15.123:0/1038251017 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408400fa30 con 0x7f4094159d20 2023-10-06T04:32:04.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f408bfff700 1 -- 172.21.15.123:0/1038251017 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408400fdd0 con 0x7f4094159d20 2023-10-06T04:32:04.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f409416b7a0 con 0x7f4094159d20 2023-10-06T04:32:04.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.907+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f409416bd60 con 0x7f4094159d20 2023-10-06T04:32:04.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.908+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40940c04e0 con 0x7f4094159d20 2023-10-06T04:32:04.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.908+0000 7f408bfff700 1 -- 172.21.15.123:0/1038251017 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4084013600 con 0x7f4094159d20 2023-10-06T04:32:04.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.909+0000 7f408bfff700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4074066f30 0x7f40740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:04.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.909+0000 7f408bfff700 1 -- 172.21.15.123:0/1038251017 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f4084084be0 con 0x7f4094159d20 2023-10-06T04:32:04.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.909+0000 7f4093fff700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4074066f30 0x7f40740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:04.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.909+0000 7f4093fff700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4074066f30 0x7f40740693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f407c0091e0 tx=0x7f407c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:04.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:04.913+0000 7f408bfff700 1 -- 172.21.15.123:0/1038251017 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f40940c04e0 con 0x7f4094159d20 2023-10-06T04:32:05.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.093+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f40940c04e0 con 0x7f4094159d20 2023-10-06T04:32:05.096 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.093+0000 7f408bfff700 1 -- 172.21.15.123:0/1038251017 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f40940c04e0 con 0x7f4094159d20 2023-10-06T04:32:05.096 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.095+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4074066f30 msgr2=0x7f40740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.095+0000 7f409a373700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4074066f30 0x7f40740693e0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f407c0091e0 tx=0x7f407c01a000 comp rx=0 tx=0).stop 2023-10-06T04:32:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4094159d20 msgr2=0x7f40941581c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4094159d20 0x7f40941581c0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f4084002b00 tx=0x7f4084007d10 comp rx=0 tx=0).stop 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 shutdown_connections 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4074066f30 0x7f40740693e0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4094152da0 0x7f4094067ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4080003a90 0x7f409416aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 --2- 172.21.15.123:0/1038251017 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4094159d20 0x7f40941581c0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 >> 172.21.15.123:0/1038251017 conn(0x7f40940c9ca0 msgr2=0x7f40940c7ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 shutdown_connections 2023-10-06T04:32:05.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.096+0000 7f409a373700 1 -- 172.21.15.123:0/1038251017 wait complete. 2023-10-06T04:32:05.110 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:32:05.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.228+0000 7fe908b4d700 1 Processor -- start 2023-10-06T04:32:05.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.228+0000 7fe908b4d700 1 -- start start 2023-10-06T04:32:05.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe908b4d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9040688c0 0x7fe9040ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe908b4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040abc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe908b4d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9040c6de0 con 0x7fe9040c7e70 2023-10-06T04:32:05.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe908b4d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9040ac190 con 0x7fe9040688c0 2023-10-06T04:32:05.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe908b4d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9040c70c0 con 0x7fe904068c90 2023-10-06T04:32:05.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe90259c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9040688c0 0x7fe9040ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe902d9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe9040c7e70 0x7fe9040c82f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53180/0 (socket says 172.21.15.123:53180) 2023-10-06T04:32:05.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe90259c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe9040688c0 0x7fe9040ab710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42212/0 (socket says 172.21.15.123:42212) 2023-10-06T04:32:05.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe902d9d700 1 -- 172.21.15.123:0/3015374219 learned_addr learned my addr 172.21.15.123:0/3015374219 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:05.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.229+0000 7fe901d9b700 1 --2- 172.21.15.123:0/3015374219 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040abc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90259c700 1 -- 172.21.15.123:0/3015374219 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 msgr2=0x7fe9040abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:05.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90259c700 1 --2- 172.21.15.123:0/3015374219 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040abc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90259c700 1 -- 172.21.15.123:0/3015374219 >> v1:172.21.15.123:6789/0 conn(0x7fe9040c7e70 legacy=0x7fe9040c82f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90259c700 1 -- 172.21.15.123:0/3015374219 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9040c73a0 con 0x7fe9040688c0 2023-10-06T04:32:05.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90159a700 1 -- 172.21.15.123:0/3015374219 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4043828560 0 0) 0x7fe9040c6de0 con 0x7fe9040c7e70 2023-10-06T04:32:05.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90259c700 1 --2- 172.21.15.123:0/3015374219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040688c0 0x7fe9040ab710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80098e0 tx=0x7fe8f80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d89843328669b2c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90159a700 1 -- 172.21.15.123:0/3015374219 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f8009e80 con 0x7fe9040688c0 2023-10-06T04:32:05.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90159a700 1 -- 172.21.15.123:0/3015374219 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8f800b860 con 0x7fe9040688c0 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe90159a700 1 -- 172.21.15.123:0/3015374219 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8f800bbe0 con 0x7fe9040688c0 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe908b4d700 1 -- 172.21.15.123:0/3015374219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040688c0 msgr2=0x7fe9040ab710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.230+0000 7fe908b4d700 1 --2- 172.21.15.123:0/3015374219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040688c0 0x7fe9040ab710 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80098e0 tx=0x7fe8f80091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.231+0000 7fe908b4d700 1 -- 172.21.15.123:0/3015374219 shutdown_connections 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.231+0000 7fe908b4d700 1 --2- 172.21.15.123:0/3015374219 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040688c0 0x7fe9040ab710 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.231+0000 7fe908b4d700 1 --2- 172.21.15.123:0/3015374219 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040abc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.231+0000 7fe908b4d700 1 -- 172.21.15.123:0/3015374219 >> 172.21.15.123:0/3015374219 conn(0x7fe9040c9ca0 msgr2=0x7fe90406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.231+0000 7fe908b4d700 1 -- 172.21.15.123:0/3015374219 shutdown_connections 2023-10-06T04:32:05.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.231+0000 7fe908b4d700 1 -- 172.21.15.123:0/3015374219 wait complete. 2023-10-06T04:32:05.239 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.232+0000 7fe908b4d700 1 Processor -- start 2023-10-06T04:32:05.239 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.232+0000 7fe908b4d700 1 -- start start 2023-10-06T04:32:05.239 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe908b4d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9040688c0 0x7fe9040c2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.239 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe908b4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040b1b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.239 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe908b4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040c7e70 0x7fe904147d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.239 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe908b4d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90414d4b0 con 0x7fe9040c7e70 2023-10-06T04:32:05.239 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe908b4d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe90414d790 con 0x7fe9040688c0 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe908b4d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe90414da70 con 0x7fe904068c90 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040c7e70 0x7fe904147d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040c7e70 0x7fe904147d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42216/0 (socket says 172.21.15.123:42216) 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 -- 172.21.15.123:0/4240062893 learned_addr learned my addr 172.21.15.123:0/4240062893 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe901d9b700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040b1b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe90259c700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9040688c0 0x7fe9040c2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 -- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 msgr2=0x7fe9040b1b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040b1b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 -- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9040688c0 msgr2=0x7fe9040c2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9040688c0 0x7fe9040c2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe902d9d700 1 -- 172.21.15.123:0/4240062893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8f8009420 con 0x7fe9040c7e70 2023-10-06T04:32:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe901d9b700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.233+0000 7fe90259c700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9040688c0 0x7fe9040c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.234+0000 7fe902d9d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040c7e70 0x7fe904147d00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00cbf0 tx=0x7fe8fc0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.234+0000 7fe8f27fc700 1 -- 172.21.15.123:0/4240062893 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc00a9c0 con 0x7fe9040c7e70 2023-10-06T04:32:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.234+0000 7fe8f27fc700 1 -- 172.21.15.123:0/4240062893 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8fc00ab20 con 0x7fe9040c7e70 2023-10-06T04:32:05.241 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.234+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe90414dd50 con 0x7fe9040c7e70 2023-10-06T04:32:05.242 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.234+0000 7fe8f27fc700 1 -- 172.21.15.123:0/4240062893 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8fc0070f0 con 0x7fe9040c7e70 2023-10-06T04:32:05.242 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.234+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe90414d160 con 0x7fe9040c7e70 2023-10-06T04:32:05.242 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.235+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe90414df40 con 0x7fe9040c7e70 2023-10-06T04:32:05.242 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.235+0000 7fe8f27fc700 1 -- 172.21.15.123:0/4240062893 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8fc007520 con 0x7fe9040c7e70 2023-10-06T04:32:05.242 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.235+0000 7fe8f27fc700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8e4066f30 0x7fe8e40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.236+0000 7fe8f27fc700 1 -- 172.21.15.123:0/4240062893 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fe8fc084a90 con 0x7fe9040c7e70 2023-10-06T04:32:05.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.236+0000 7fe90259c700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8e4066f30 0x7fe8e40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.236+0000 7fe90259c700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8e4066f30 0x7fe8e40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80091e0 tx=0x7fe8f8017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.240+0000 7fe8f27fc700 1 -- 172.21.15.123:0/4240062893 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8fc051c20 con 0x7fe9040c7e70 2023-10-06T04:32:05.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.422+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe90414d5f0 con 0x7fe9040c7e70 2023-10-06T04:32:05.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.423+0000 7fe8f27fc700 1 -- 172.21.15.123:0/4240062893 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe90414d5f0 con 0x7fe9040c7e70 2023-10-06T04:32:05.425 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:05.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8e4066f30 msgr2=0x7fe8e40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8e4066f30 0x7fe8e40693e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fe8f80091e0 tx=0x7fe8f8017000 comp rx=0 tx=0).stop 2023-10-06T04:32:05.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040c7e70 msgr2=0x7fe904147d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040c7e70 0x7fe904147d00 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fe8fc00cbf0 tx=0x7fe8fc0019b0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 shutdown_connections 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8e4066f30 0x7fe8e40693e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe9040c7e70 0x7fe904147d00 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe9040688c0 0x7fe9040c2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 --2- 172.21.15.123:0/4240062893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe904068c90 0x7fe9040b1b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 >> 172.21.15.123:0/4240062893 conn(0x7fe9040c9ca0 msgr2=0x7fe904069aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 shutdown_connections 2023-10-06T04:32:05.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.425+0000 7fe908b4d700 1 -- 172.21.15.123:0/4240062893 wait complete. 2023-10-06T04:32:05.439 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:32:05.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.556+0000 7f915a295700 1 Processor -- start 2023-10-06T04:32:05.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.556+0000 7f915a295700 1 -- start start 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f915a295700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9154157080 0x7f91540abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f915a295700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91540ac470 0x7f9154068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f915a295700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91540c7810 con 0x7f9154157450 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f915a295700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91540ac9f0 con 0x7f9154157080 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f915a295700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91540c6ee0 con 0x7f91540ac470 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f91537fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91540ac470 0x7f9154068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f9153fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9154157080 0x7f91540abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.561 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f91537fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91540ac470 0x7f9154068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:37612/0 (socket says 172.21.15.123:37612) 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f9153fff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9154157080 0x7f91540abf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:48774/0 (socket says 172.21.15.123:48774) 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f91537fe700 1 -- 172.21.15.123:0/3567172642 learned_addr learned my addr 172.21.15.123:0/3567172642 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f9153fff700 1 -- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91540ac470 msgr2=0x7f9154068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f9153fff700 1 --2- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91540ac470 0x7f9154068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.557+0000 7f9153fff700 1 -- 172.21.15.123:0/3567172642 >> v1:172.21.15.177:6789/0 conn(0x7f9154157450 legacy=0x7f91540ade60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f9153fff700 1 -- 172.21.15.123:0/3567172642 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91540c71c0 con 0x7f9154157080 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f9152ffd700 1 -- 172.21.15.123:0/3567172642 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1910176749 0 0) 0x7f91540c7810 con 0x7f9154157450 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f91537fe700 1 --2- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91540ac470 0x7f9154068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f9153fff700 1 --2- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9154157080 0x7f91540abf30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f91480098e0 tx=0x7f91480091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3c01be843cffa0a7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f9152ffd700 1 -- 172.21.15.123:0/3567172642 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800b640 con 0x7f9154157080 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f9152ffd700 1 -- 172.21.15.123:0/3567172642 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914800b7a0 con 0x7f9154157080 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f9152ffd700 1 -- 172.21.15.123:0/3567172642 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914800bb40 con 0x7f9154157080 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f915a295700 1 -- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9154157080 msgr2=0x7f91540abf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f915a295700 1 --2- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9154157080 0x7f91540abf30 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f91480098e0 tx=0x7f91480091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f915a295700 1 -- 172.21.15.123:0/3567172642 shutdown_connections 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f915a295700 1 --2- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9154157080 0x7f91540abf30 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f915a295700 1 --2- 172.21.15.123:0/3567172642 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f91540ac470 0x7f9154068670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f915a295700 1 -- 172.21.15.123:0/3567172642 >> 172.21.15.123:0/3567172642 conn(0x7f91540c9ca0 msgr2=0x7f915406da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.558+0000 7f915a295700 1 -- 172.21.15.123:0/3567172642 shutdown_connections 2023-10-06T04:32:05.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.559+0000 7f915a295700 1 -- 172.21.15.123:0/3567172642 wait complete. 2023-10-06T04:32:05.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 Processor -- start 2023-10-06T04:32:05.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 -- start start 2023-10-06T04:32:05.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91540ac470 0x7f91540af630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9154157080 0x7f91540b1120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9154157450 0x7f9154151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91540699a0 con 0x7f9154157080 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91540afb70 con 0x7f91540ac470 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.560+0000 7f915a295700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f915416d3e0 con 0x7f9154157450 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9154157080 0x7f91540b1120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9154157080 0x7f91540b1120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42228/0 (socket says 172.21.15.123:42228) 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 -- 172.21.15.123:0/2516978346 learned_addr learned my addr 172.21.15.123:0/2516978346 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f9153fff700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91540ac470 0x7f91540af630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f9158832700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9154157450 0x7f9154151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 -- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9154157450 msgr2=0x7f9154151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9154157450 0x7f9154151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 -- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91540ac470 msgr2=0x7f91540af630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91540ac470 0x7f91540af630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 -- 172.21.15.123:0/2516978346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91440094f0 con 0x7f9154157080 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f91537fe700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9154157080 0x7f91540b1120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f914400ec50 tx=0x7f9144009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f913bfff700 1 -- 172.21.15.123:0/2516978346 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914400c850 con 0x7f9154157080 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f913bfff700 1 -- 172.21.15.123:0/2516978346 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f914400c9b0 con 0x7f9154157080 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9148009420 con 0x7f9154157080 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f913bfff700 1 -- 172.21.15.123:0/2516978346 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f914400cd50 con 0x7f9154157080 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.561+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f915416d9a0 con 0x7f9154157080 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.562+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9154069b50 con 0x7f9154157080 2023-10-06T04:32:05.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.563+0000 7f913bfff700 1 -- 172.21.15.123:0/2516978346 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9144013680 con 0x7f9154157080 2023-10-06T04:32:05.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.563+0000 7f913bfff700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9140066f30 0x7f91400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.563+0000 7f913bfff700 1 -- 172.21.15.123:0/2516978346 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9144084ce0 con 0x7f9154157080 2023-10-06T04:32:05.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.563+0000 7f9153fff700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9140066f30 0x7f91400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.563+0000 7f9153fff700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9140066f30 0x7f91400693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91480091e0 tx=0x7f9148017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.567+0000 7f913bfff700 1 -- 172.21.15.123:0/2516978346 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9154069b50 con 0x7f9154157080 2023-10-06T04:32:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.747+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9154069b50 con 0x7f9154157080 2023-10-06T04:32:05.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.748+0000 7f913bfff700 1 -- 172.21.15.123:0/2516978346 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9154069b50 con 0x7f9154157080 2023-10-06T04:32:05.750 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9140066f30 msgr2=0x7f91400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9140066f30 0x7f91400693e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f91480091e0 tx=0x7f9148017000 comp rx=0 tx=0).stop 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9154157080 msgr2=0x7f91540b1120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9154157080 0x7f91540b1120 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f914400ec50 tx=0x7f9144009250 comp rx=0 tx=0).stop 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 shutdown_connections 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9140066f30 0x7f91400693e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9154157450 0x7f9154151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f91540ac470 0x7f91540af630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 --2- 172.21.15.123:0/2516978346 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9154157080 0x7f91540b1120 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 >> 172.21.15.123:0/2516978346 conn(0x7f91540c9ca0 msgr2=0x7f91540c8530 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:05.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 shutdown_connections 2023-10-06T04:32:05.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.750+0000 7f915a295700 1 -- 172.21.15.123:0/2516978346 wait complete. 2023-10-06T04:32:05.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:05.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.879+0000 7efe4f6e2700 1 Processor -- start 2023-10-06T04:32:05.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.879+0000 7efe4f6e2700 1 -- start start 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4f6e2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe48156f60 0x7efe4815ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4f6e2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efe480ade40 0x7efe48157da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4f6e2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe48157910 con 0x7efe4815b290 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4f6e2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4806d240 con 0x7efe48156f60 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4f6e2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe4806d520 con 0x7efe480ade40 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4d47e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe48156f60 0x7efe4815ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4cc7d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efe4815b290 0x7efe480ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53196/0 (socket says 172.21.15.123:53196) 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4cc7d700 1 -- 172.21.15.123:0/4265373725 learned_addr learned my addr 172.21.15.123:0/4265373725 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4d47e700 1 -- 172.21.15.123:0/4265373725 >> v2:172.21.15.177:3300/0 conn(0x7efe480ade40 msgr2=0x7efe48157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4dc7f700 1 --2- 172.21.15.123:0/4265373725 >> v2:172.21.15.177:3300/0 conn(0x7efe480ade40 0x7efe48157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4d47e700 1 --2- 172.21.15.123:0/4265373725 >> v2:172.21.15.177:3300/0 conn(0x7efe480ade40 0x7efe48157da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4d47e700 1 -- 172.21.15.123:0/4265373725 >> v1:172.21.15.123:6789/0 conn(0x7efe4815b290 legacy=0x7efe480ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe3ffff700 1 -- 172.21.15.123:0/4265373725 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898395774 0 0) 0x7efe48157910 con 0x7efe4815b290 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.880+0000 7efe4d47e700 1 -- 172.21.15.123:0/4265373725 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe4806d800 con 0x7efe48156f60 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4d47e700 1 --2- 172.21.15.123:0/4265373725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe48156f60 0x7efe4815ad50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe440098e0 tx=0x7efe440091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ac1ee0a58f8a36 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe3ffff700 1 -- 172.21.15.123:0/4265373725 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe44009e80 con 0x7efe48156f60 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe3ffff700 1 -- 172.21.15.123:0/4265373725 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe4400b820 con 0x7efe48156f60 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe3ffff700 1 -- 172.21.15.123:0/4265373725 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4400bba0 con 0x7efe48156f60 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 -- 172.21.15.123:0/4265373725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe48156f60 msgr2=0x7efe4815ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/4265373725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe48156f60 0x7efe4815ad50 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7efe440098e0 tx=0x7efe440091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 -- 172.21.15.123:0/4265373725 shutdown_connections 2023-10-06T04:32:05.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/4265373725 >> v2:172.21.15.177:3300/0 conn(0x7efe480ade40 0x7efe48157da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/4265373725 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe48156f60 0x7efe4815ad50 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 -- 172.21.15.123:0/4265373725 >> 172.21.15.123:0/4265373725 conn(0x7efe480ce0c0 msgr2=0x7efe480ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:05.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 -- 172.21.15.123:0/4265373725 shutdown_connections 2023-10-06T04:32:05.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.881+0000 7efe4f6e2700 1 -- 172.21.15.123:0/4265373725 wait complete. 2023-10-06T04:32:05.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.882+0000 7efe4f6e2700 1 Processor -- start 2023-10-06T04:32:05.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.882+0000 7efe4f6e2700 1 -- start start 2023-10-06T04:32:05.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4f6e2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe480ade40 0x7efe481688c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4f6e2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe48156f60 0x7efe480b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4f6e2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe4815b290 0x7efe481670e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4f6e2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe48167620 con 0x7efe4815b290 2023-10-06T04:32:05.890 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4f6e2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe480b1ff0 con 0x7efe480ade40 2023-10-06T04:32:05.890 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4f6e2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe480b22d0 con 0x7efe48156f60 2023-10-06T04:32:05.890 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4dc7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe4815b290 0x7efe481670e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.890 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4dc7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe4815b290 0x7efe481670e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42242/0 (socket says 172.21.15.123:42242) 2023-10-06T04:32:05.890 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.883+0000 7efe4dc7f700 1 -- 172.21.15.123:0/3112123668 learned_addr learned my addr 172.21.15.123:0/3112123668 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4d47e700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe480ade40 0x7efe481688c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4dc7f700 1 -- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe48156f60 msgr2=0x7efe480b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4cc7d700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe48156f60 0x7efe480b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4dc7f700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe48156f60 0x7efe480b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4dc7f700 1 -- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe480ade40 msgr2=0x7efe481688c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4dc7f700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe480ade40 0x7efe481688c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4dc7f700 1 -- 172.21.15.123:0/3112123668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe44009420 con 0x7efe4815b290 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4d47e700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe480ade40 0x7efe481688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4cc7d700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe48156f60 0x7efe480b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:05.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4dc7f700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe4815b290 0x7efe481670e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe4000ce50 tx=0x7efe40007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe3d7fa700 1 -- 172.21.15.123:0/3112123668 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe4000ae40 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe3d7fa700 1 -- 172.21.15.123:0/3112123668 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe40016430 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe48158f80 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe3d7fa700 1 -- 172.21.15.123:0/3112123668 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe400167d0 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.884+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe48159540 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.885+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe48159760 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.885+0000 7efe3d7fa700 1 -- 172.21.15.123:0/3112123668 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe40003020 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.886+0000 7efe3d7fa700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe2c066f30 0x7efe2c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.886+0000 7efe4d47e700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe2c066f30 0x7efe2c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.886+0000 7efe3d7fa700 1 -- 172.21.15.123:0/3112123668 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7efe40012070 con 0x7efe4815b290 2023-10-06T04:32:05.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.886+0000 7efe4d47e700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe2c066f30 0x7efe2c0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe44003080 tx=0x7efe44017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:05.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:05.890+0000 7efe3d7fa700 1 -- 172.21.15.123:0/3112123668 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe40052770 con 0x7efe4815b290 2023-10-06T04:32:06.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.077+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe4806dae0 con 0x7efe4815b290 2023-10-06T04:32:06.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.078+0000 7efe3d7fa700 1 -- 172.21.15.123:0/3112123668 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7efe40055d90 con 0x7efe4815b290 2023-10-06T04:32:06.081 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:06.081 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe2c066f30 msgr2=0x7efe2c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe2c066f30 0x7efe2c0693e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7efe44003080 tx=0x7efe44017000 comp rx=0 tx=0).stop 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe4815b290 msgr2=0x7efe481670e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe4815b290 0x7efe481670e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7efe4000ce50 tx=0x7efe40007810 comp rx=0 tx=0).stop 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 shutdown_connections 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe2c066f30 0x7efe2c0693e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe4815b290 0x7efe481670e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe480ade40 0x7efe481688c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 --2- 172.21.15.123:0/3112123668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe48156f60 0x7efe480b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 >> 172.21.15.123:0/3112123668 conn(0x7efe480ce0c0 msgr2=0x7efe480cc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:06.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 shutdown_connections 2023-10-06T04:32:06.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.080+0000 7efe4f6e2700 1 -- 172.21.15.123:0/3112123668 wait complete. 2023-10-06T04:32:06.094 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:31:32.045143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:31:33.390421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:32:06.095 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:06.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.212+0000 7fa36a2cc700 1 Processor -- start 2023-10-06T04:32:06.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.212+0000 7fa36a2cc700 1 -- start start 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa36a2cc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413fdc0 0x7fa364067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa36a2cc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa364067ba0 0x7fa3640c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa36a2cc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa36406b020 con 0x7fa36413f9f0 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa36a2cc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa364067ff0 con 0x7fa364067ba0 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa36a2cc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36406a450 con 0x7fa36413fdc0 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa3637fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413fdc0 0x7fa364067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa368869700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa364067ba0 0x7fa3640c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa3637fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413fdc0 0x7fa364067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:37636/0 (socket says 172.21.15.123:37636) 2023-10-06T04:32:06.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa3637fe700 1 -- 172.21.15.123:0/1404350994 learned_addr learned my addr 172.21.15.123:0/1404350994 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.213+0000 7fa368869700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa364067ba0 0x7fa3640c7660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:48806/0 (socket says 172.21.15.123:48806) 2023-10-06T04:32:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa3637fe700 1 -- 172.21.15.123:0/1404350994 >> v2:172.21.15.177:3300/0 conn(0x7fa364067ba0 msgr2=0x7fa3640c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa3637fe700 1 --2- 172.21.15.123:0/1404350994 >> v2:172.21.15.177:3300/0 conn(0x7fa364067ba0 0x7fa3640c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa3637fe700 1 -- 172.21.15.123:0/1404350994 >> v1:172.21.15.177:6789/0 conn(0x7fa36413f9f0 legacy=0x7fa36406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa3637fe700 1 -- 172.21.15.123:0/1404350994 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa36406bbc0 con 0x7fa36413fdc0 2023-10-06T04:32:06.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa3637fe700 1 --2- 172.21.15.123:0/1404350994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413fdc0 0x7fa364067660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3540099b0 tx=0x7fa354009250 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c790af31397c145b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa362ffd700 1 -- 172.21.15.123:0/1404350994 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa354009ea0 con 0x7fa36413fdc0 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa362ffd700 1 -- 172.21.15.123:0/1404350994 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35400b860 con 0x7fa36413fdc0 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa362ffd700 1 -- 172.21.15.123:0/1404350994 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35400bc00 con 0x7fa36413fdc0 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 -- 172.21.15.123:0/1404350994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413fdc0 msgr2=0x7fa364067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/1404350994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413fdc0 0x7fa364067660 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa3540099b0 tx=0x7fa354009250 comp rx=0 tx=0).stop 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 -- 172.21.15.123:0/1404350994 shutdown_connections 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/1404350994 >> v2:172.21.15.177:3300/0 conn(0x7fa364067ba0 0x7fa3640c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/1404350994 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413fdc0 0x7fa364067660 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 -- 172.21.15.123:0/1404350994 >> 172.21.15.123:0/1404350994 conn(0x7fa3640cd020 msgr2=0x7fa3640cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 -- 172.21.15.123:0/1404350994 shutdown_connections 2023-10-06T04:32:06.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.214+0000 7fa36a2cc700 1 -- 172.21.15.123:0/1404350994 wait complete. 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.215+0000 7fa36a2cc700 1 Processor -- start 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.216+0000 7fa36a2cc700 1 -- start start 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.216+0000 7fa36a2cc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa364067ba0 0x7fa36413dd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.216+0000 7fa36a2cc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413f9f0 0x7fa36415a830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.216+0000 7fa36a2cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36416ab30 0x7fa36415b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.216+0000 7fa36a2cc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415d890 con 0x7fa36416ab30 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.216+0000 7fa36a2cc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415ccc0 con 0x7fa364067ba0 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.216+0000 7fa36a2cc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa36415cf60 con 0x7fa36413f9f0 2023-10-06T04:32:06.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36416ab30 0x7fa36415b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa363fff700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa364067ba0 0x7fa36413dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa3637fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413f9f0 0x7fa36415a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36416ab30 0x7fa36415b1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42248/0 (socket says 172.21.15.123:42248) 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 -- 172.21.15.123:0/3833418365 learned_addr learned my addr 172.21.15.123:0/3833418365 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 -- 172.21.15.123:0/3833418365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413f9f0 msgr2=0x7fa36415a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413f9f0 0x7fa36415a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 -- 172.21.15.123:0/3833418365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa364067ba0 msgr2=0x7fa36413dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa364067ba0 0x7fa36413dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 -- 172.21.15.123:0/3833418365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3540094f0 con 0x7fa36416ab30 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa368869700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36416ab30 0x7fa36415b1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009df0 tx=0x7fa35c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa34bfff700 1 -- 172.21.15.123:0/3833418365 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00caa0 con 0x7fa36416ab30 2023-10-06T04:32:06.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa34bfff700 1 -- 172.21.15.123:0/3833418365 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa35c00f040 con 0x7fa36416ab30 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa36415d240 con 0x7fa36416ab30 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa34bfff700 1 -- 172.21.15.123:0/3833418365 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa35c00ce70 con 0x7fa36416ab30 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.217+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa36415ad70 con 0x7fa36416ab30 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.218+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa36415b020 con 0x7fa36416ab30 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.219+0000 7fa34bfff700 1 -- 172.21.15.123:0/3833418365 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa35c014690 con 0x7fa36416ab30 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.219+0000 7fa34bfff700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.219+0000 7fa34bfff700 1 -- 172.21.15.123:0/3833418365 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa35c084f20 con 0x7fa36416ab30 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.219+0000 7fa363fff700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.219+0000 7fa363fff700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa350066f30 0x7fa3500693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa36415a010 tx=0x7fa358009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.223+0000 7fa34bfff700 1 -- 172.21.15.123:0/3833418365 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa36415b020 con 0x7fa36416ab30 2023-10-06T04:32:06.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.406+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa36415b020 con 0x7fa36416ab30 2023-10-06T04:32:06.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.406+0000 7fa34bfff700 1 -- 172.21.15.123:0/3833418365 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa35c018020 con 0x7fa36416ab30 2023-10-06T04:32:06.409 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa350066f30 msgr2=0x7fa3500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa350066f30 0x7fa3500693e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fa36415a010 tx=0x7fa358009040 comp rx=0 tx=0).stop 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36416ab30 msgr2=0x7fa36415b1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36416ab30 0x7fa36415b1c0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7fa35c009df0 tx=0x7fa35c00b040 comp rx=0 tx=0).stop 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 shutdown_connections 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa350066f30 0x7fa3500693e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa36416ab30 0x7fa36415b1c0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa364067ba0 0x7fa36413dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 --2- 172.21.15.123:0/3833418365 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa36413f9f0 0x7fa36415a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.412 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 >> 172.21.15.123:0/3833418365 conn(0x7fa3640cd020 msgr2=0x7fa36413ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:06.412 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 shutdown_connections 2023-10-06T04:32:06.412 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.408+0000 7fa36a2cc700 1 -- 172.21.15.123:0/3833418365 wait complete. 2023-10-06T04:32:06.422 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:06.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.539+0000 7fa7cf5b5700 1 Processor -- start 2023-10-06T04:32:06.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.539+0000 7fa7cf5b5700 1 -- start start 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cf5b5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa7c8068d80 0x7fa7c80c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cf5b5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c8069aa0 con 0x7fa7c80c8710 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cf5b5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7c806a340 con 0x7fa7c80c8340 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cf5b5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7c80aa500 con 0x7fa7c8068d80 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cdb52700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa7c8068d80 0x7fa7c80c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7ccb50700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa7c80c8710 0x7fa7c8068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53202/0 (socket says 172.21.15.123:53202) 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7ccb50700 1 -- 172.21.15.123:0/2911658400 learned_addr learned my addr 172.21.15.123:0/2911658400 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cdb52700 1 -- 172.21.15.123:0/2911658400 >> v1:172.21.15.177:6789/0 conn(0x7fa7c80c8340 legacy=0x7fa7c80a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cdb52700 1 -- 172.21.15.123:0/2911658400 >> v1:172.21.15.123:6789/0 conn(0x7fa7c80c8710 legacy=0x7fa7c8068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7cdb52700 1 -- 172.21.15.123:0/2911658400 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c80aa6f0 con 0x7fa7c8068d80 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.540+0000 7fa7bffff700 1 -- 172.21.15.123:0/2911658400 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 6739650 0 0) 0x7fa7c8069aa0 con 0x7fa7c80c8710 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cdb52700 1 --2- 172.21.15.123:0/2911658400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c8b60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009e60 tx=0x7fa7c0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13b3e0f4be1cbbaf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7bffff700 1 -- 172.21.15.123:0/2911658400 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c000e070 con 0x7fa7c8068d80 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7bffff700 1 -- 172.21.15.123:0/2911658400 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c000bc30 con 0x7fa7c8068d80 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7bffff700 1 -- 172.21.15.123:0/2911658400 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c0012430 con 0x7fa7c8068d80 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2911658400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 msgr2=0x7fa7c80c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2911658400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c8b60 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa7c0009e60 tx=0x7fa7c0009780 comp rx=0 tx=0).stop 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2911658400 shutdown_connections 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2911658400 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c8b60 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2911658400 >> 172.21.15.123:0/2911658400 conn(0x7fa7c80c9ca0 msgr2=0x7fa7c806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2911658400 shutdown_connections 2023-10-06T04:32:06.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.541+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2911658400 wait complete. 2023-10-06T04:32:06.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.542+0000 7fa7cf5b5700 1 Processor -- start 2023-10-06T04:32:06.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.542+0000 7fa7cf5b5700 1 -- start start 2023-10-06T04:32:06.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cf5b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cf5b5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80c8340 0x7fa7c815c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cf5b5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80c8710 0x7fa7c8156a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cf5b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c80c2f30 con 0x7fa7c8068d80 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cf5b5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c80c2560 con 0x7fa7c80c8340 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cf5b5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c80c1990 con 0x7fa7c80c8710 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cd351700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cd351700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c29f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42258/0 (socket says 172.21.15.123:42258) 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7cd351700 1 -- 172.21.15.123:0/2549189736 learned_addr learned my addr 172.21.15.123:0/2549189736 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.543+0000 7fa7ccb50700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80c8340 0x7fa7c815c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cdb52700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80c8710 0x7fa7c8156a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cd351700 1 -- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80c8710 msgr2=0x7fa7c8156a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cd351700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80c8710 0x7fa7c8156a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cd351700 1 -- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80c8340 msgr2=0x7fa7c815c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.549 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cd351700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80c8340 0x7fa7c815c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cd351700 1 -- 172.21.15.123:0/2549189736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c0009a20 con 0x7fa7c8068d80 2023-10-06T04:32:06.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cdb52700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80c8710 0x7fa7c8156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:06.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cd351700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c29f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800c7c0 tx=0x7fa7b8007f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7bd7fa700 1 -- 172.21.15.123:0/2549189736 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8011590 con 0x7fa7c8068d80 2023-10-06T04:32:06.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7bd7fa700 1 -- 172.21.15.123:0/2549189736 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7b8011710 con 0x7fa7c8068d80 2023-10-06T04:32:06.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7c80c1c30 con 0x7fa7c8068d80 2023-10-06T04:32:06.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7bd7fa700 1 -- 172.21.15.123:0/2549189736 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7b8011ad0 con 0x7fa7c8068d80 2023-10-06T04:32:06.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.544+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7c80c21f0 con 0x7fa7c8068d80 2023-10-06T04:32:06.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.545+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7c8069cf0 con 0x7fa7c8068d80 2023-10-06T04:32:06.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.545+0000 7fa7bd7fa700 1 -- 172.21.15.123:0/2549189736 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa7b801b3b0 con 0x7fa7c8068d80 2023-10-06T04:32:06.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.546+0000 7fa7bd7fa700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa7ac066f30 0x7fa7ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.546+0000 7fa7bd7fa700 1 -- 172.21.15.123:0/2549189736 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fa7b8083ac0 con 0x7fa7c8068d80 2023-10-06T04:32:06.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.546+0000 7fa7ccb50700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa7ac066f30 0x7fa7ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.546+0000 7fa7ccb50700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa7ac066f30 0x7fa7ac0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7c815e630 tx=0x7fa7c4007f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.551+0000 7fa7bd7fa700 1 -- 172.21.15.123:0/2549189736 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa7b8050bd0 con 0x7fa7c8068d80 2023-10-06T04:32:06.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.730+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7fa7c80c2840 con 0x7fa7c8068d80 2023-10-06T04:32:06.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.731+0000 7fa7bd7fa700 1 -- 172.21.15.123:0/2549189736 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7fa7b8017070 con 0x7fa7c8068d80 2023-10-06T04:32:06.733 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa7ac066f30 msgr2=0x7fa7ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa7ac066f30 0x7fa7ac0693e0 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fa7c815e630 tx=0x7fa7c4007f50 comp rx=0 tx=0).stop 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 msgr2=0x7fa7c80c29f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c29f0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa7b800c7c0 tx=0x7fa7b8007f40 comp rx=0 tx=0).stop 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 shutdown_connections 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa7ac066f30 0x7fa7ac0693e0 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa7c8068d80 0x7fa7c80c29f0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa7c80c8710 0x7fa7c8156a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 --2- 172.21.15.123:0/2549189736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa7c80c8340 0x7fa7c815c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 >> 172.21.15.123:0/2549189736 conn(0x7fa7c80c9ca0 msgr2=0x7fa7c80b1f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:06.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 shutdown_connections 2023-10-06T04:32:06.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.733+0000 7fa7cf5b5700 1 -- 172.21.15.123:0/2549189736 wait complete. 2023-10-06T04:32:06.747 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:32:06.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.865+0000 7f25517d6700 1 Processor -- start 2023-10-06T04:32:06.867 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.865+0000 7f25517d6700 1 -- start start 2023-10-06T04:32:06.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.865+0000 7f25517d6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f254c06e300 0x7f254c159950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.873 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.865+0000 7f25517d6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c06e7c0 con 0x7f254c159e90 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.865+0000 7f25517d6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c06eaa0 con 0x7f254c0af440 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.865+0000 7f25517d6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c15d040 con 0x7f254c06e300 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f254b7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f254c159e90 0x7f254c067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53212/0 (socket says 172.21.15.123:53212) 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f254b7fe700 1 -- 172.21.15.123:0/4070220151 learned_addr learned my addr 172.21.15.123:0/4070220151 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f254a7fc700 1 --2- 172.21.15.123:0/4070220151 >> v2:172.21.15.177:3300/0 conn(0x7f254c06e300 0x7f254c159950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2741617477 0 0) 0x7f254c06e7c0 con 0x7f254c159e90 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2538003570 con 0x7f254c159e90 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1254690315 0 0) 0x7f2538003570 con 0x7f254c159e90 2023-10-06T04:32:06.874 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 >> v2:172.21.15.177:3300/0 conn(0x7f254c06e300 msgr2=0x7f254c159950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 --2- 172.21.15.123:0/4070220151 >> v2:172.21.15.177:3300/0 conn(0x7f254c06e300 0x7f254c159950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 >> v1:172.21.15.177:6789/0 conn(0x7f254c0af440 legacy=0x7f254c0c3930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c06ed80 con 0x7f254c159e90 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f25440017f0 con 0x7f254c159e90 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 >> v1:172.21.15.123:6789/0 conn(0x7f254c159e90 legacy=0x7f254c067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2538003a60 0x7f2538003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2538004810 0x7f2538004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25380051c0 0x7f253800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.875 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c15d040 con 0x7f25380051c0 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253800c0d0 con 0x7f2538003a60 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253800c290 con 0x7f2538004810 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f254b7fe700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25380051c0 0x7f253800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f254affd700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2538003a60 0x7f2538003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.866+0000 7f254a7fc700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2538004810 0x7f2538004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f254b7fe700 1 -- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2538004810 msgr2=0x7f2538004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f254b7fe700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2538004810 0x7f2538004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f254b7fe700 1 -- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2538003a60 msgr2=0x7f2538003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:06.876 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f254b7fe700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2538003a60 0x7f2538003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f254b7fe700 1 -- 172.21.15.123:0/4070220151 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2538003570 con 0x7f25380051c0 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f254b7fe700 1 -- 172.21.15.123:0/4070220151 reap_dead start 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f254b7fe700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25380051c0 0x7f253800ba60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f254400a930 tx=0x7f25440021d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c76487e014b3655 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25440070d0 con 0x7f25380051c0 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2544007230 con 0x7f25380051c0 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f2549ffb700 1 -- 172.21.15.123:0/4070220151 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25440075d0 con 0x7f25380051c0 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f25517d6700 1 -- 172.21.15.123:0/4070220151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25380051c0 msgr2=0x7f253800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f25517d6700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25380051c0 0x7f253800ba60 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f254400a930 tx=0x7f25440021d0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f25517d6700 1 -- 172.21.15.123:0/4070220151 shutdown_connections 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f25517d6700 1 --2- 172.21.15.123:0/4070220151 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f25380051c0 0x7f253800ba60 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f25517d6700 1 -- 172.21.15.123:0/4070220151 >> 172.21.15.123:0/4070220151 conn(0x7f254c0cd030 msgr2=0x7f254c0cd430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f25517d6700 1 -- 172.21.15.123:0/4070220151 shutdown_connections 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.867+0000 7f25517d6700 1 -- 172.21.15.123:0/4070220151 wait complete. 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 Processor -- start 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 -- start start 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25380051c0 0x7f254c0c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f254c15ab50 0x7f254c14fb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254c157460 0x7f254c157900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c157e40 con 0x7f254c15ab50 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c15b5d0 con 0x7f254c157460 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f25517d6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c15b010 con 0x7f25380051c0 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f254a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f254c15ab50 0x7f254c14fb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f254a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f254c15ab50 0x7f254c14fb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42270/0 (socket says 172.21.15.123:42270) 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.869+0000 7f254a7fc700 1 -- 172.21.15.123:0/748627411 learned_addr learned my addr 172.21.15.123:0/748627411 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254affd700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25380051c0 0x7f254c0c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254b7fe700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254c157460 0x7f254c157900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254a7fc700 1 -- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25380051c0 msgr2=0x7f254c0c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254a7fc700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25380051c0 0x7f254c0c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254a7fc700 1 -- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254c157460 msgr2=0x7f254c157900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254a7fc700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254c157460 0x7f254c157900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254a7fc700 1 -- 172.21.15.123:0/748627411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c15b2f0 con 0x7f254c15ab50 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f254a7fc700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f254c15ab50 0x7f254c14fb00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2534009fd0 tx=0x7f25340091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f2542ffd700 1 -- 172.21.15.123:0/748627411 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400f790 con 0x7f254c15ab50 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f2542ffd700 1 -- 172.21.15.123:0/748627411 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253400f8f0 con 0x7f254c15ab50 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c0aff00 con 0x7f254c15ab50 2023-10-06T04:32:06.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f2542ffd700 1 -- 172.21.15.123:0/748627411 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253400fc70 con 0x7f254c15ab50 2023-10-06T04:32:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.870+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c0c8a70 con 0x7f254c15ab50 2023-10-06T04:32:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.871+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c06d260 con 0x7f254c15ab50 2023-10-06T04:32:06.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.871+0000 7f2542ffd700 1 -- 172.21.15.123:0/748627411 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2534014650 con 0x7f254c15ab50 2023-10-06T04:32:06.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.872+0000 7f2542ffd700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f252c066f30 0x7f252c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:06.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.872+0000 7f2542ffd700 1 -- 172.21.15.123:0/748627411 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2534018030 con 0x7f254c15ab50 2023-10-06T04:32:06.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.872+0000 7f254affd700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f252c066f30 0x7f252c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:06.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.872+0000 7f254affd700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f252c066f30 0x7f252c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f253c007e40 tx=0x7f253c007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:06.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:06.876+0000 7f2542ffd700 1 -- 172.21.15.123:0/748627411 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2534055aa0 con 0x7f254c15ab50 2023-10-06T04:32:07.061 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.059+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f254c04bb80 con 0x7f254c15ab50 2023-10-06T04:32:07.061 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.059+0000 7f2542ffd700 1 -- 172.21.15.123:0/748627411 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f2534022020 con 0x7f254c15ab50 2023-10-06T04:32:07.062 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f252c066f30 msgr2=0x7f252c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f252c066f30 0x7f252c0693e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f253c007e40 tx=0x7f253c007d30 comp rx=0 tx=0).stop 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f254c15ab50 msgr2=0x7f254c14fb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f254c15ab50 0x7f254c14fb00 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2534009fd0 tx=0x7f25340091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 shutdown_connections 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f252c066f30 0x7f252c0693e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f254c157460 0x7f254c157900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f25380051c0 0x7f254c0c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.065 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 --2- 172.21.15.123:0/748627411 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f254c15ab50 0x7f254c14fb00 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.065 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 >> 172.21.15.123:0/748627411 conn(0x7f254c0cd030 msgr2=0x7f254c15d2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:07.065 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 shutdown_connections 2023-10-06T04:32:07.065 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.061+0000 7f25517d6700 1 -- 172.21.15.123:0/748627411 wait complete. 2023-10-06T04:32:07.076 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:32:07.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.193+0000 7f72311fc700 1 Processor -- start 2023-10-06T04:32:07.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.193+0000 7f72311fc700 1 -- start start 2023-10-06T04:32:07.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f72311fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f722c0c8710 0x7f722c0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f72311fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f722c068670 0x7f722c068ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f72311fc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f722c069010 con 0x7f722c0c8340 2023-10-06T04:32:07.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f72311fc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c0aa500 con 0x7f722c0c8710 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f72311fc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f722c069aa0 con 0x7f722c068670 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f722a59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f722c0c8710 0x7f722c0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f722ad9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f722c0c8340 0x7f722c0a9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53214/0 (socket says 172.21.15.123:53214) 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f722ad9d700 1 -- 172.21.15.123:0/1243497542 learned_addr learned my addr 172.21.15.123:0/1243497542 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.194+0000 7f722b59e700 1 --2- 172.21.15.123:0/1243497542 >> v2:172.21.15.177:3300/0 conn(0x7f722c068670 0x7f722c068ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f722a59c700 1 -- 172.21.15.123:0/1243497542 >> v2:172.21.15.177:3300/0 conn(0x7f722c068670 msgr2=0x7f722c068ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f722a59c700 1 --2- 172.21.15.123:0/1243497542 >> v2:172.21.15.177:3300/0 conn(0x7f722c068670 0x7f722c068ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f722a59c700 1 -- 172.21.15.123:0/1243497542 >> v1:172.21.15.123:6789/0 conn(0x7f722c0c8340 legacy=0x7f722c0a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f722a59c700 1 -- 172.21.15.123:0/1243497542 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722c06a340 con 0x7f722c0c8710 2023-10-06T04:32:07.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f7229d9b700 1 -- 172.21.15.123:0/1243497542 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390535500 0 0) 0x7f7220001110 con 0x7f722c0c8340 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f722b59e700 1 --2- 172.21.15.123:0/1243497542 >> v2:172.21.15.177:3300/0 conn(0x7f722c068670 0x7f722c068ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f722a59c700 1 --2- 172.21.15.123:0/1243497542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0c8b60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f722c0c6cd0 tx=0x7f721c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cd1fd4f69549f59 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f7229d9b700 1 -- 172.21.15.123:0/1243497542 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c00b6b0 con 0x7f722c0c8710 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f7229d9b700 1 -- 172.21.15.123:0/1243497542 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c00b810 con 0x7f722c0c8710 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f7229d9b700 1 -- 172.21.15.123:0/1243497542 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c00bbb0 con 0x7f722c0c8710 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 -- 172.21.15.123:0/1243497542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 msgr2=0x7f722c0c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 --2- 172.21.15.123:0/1243497542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0c8b60 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f722c0c6cd0 tx=0x7f721c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 -- 172.21.15.123:0/1243497542 shutdown_connections 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 --2- 172.21.15.123:0/1243497542 >> v2:172.21.15.177:3300/0 conn(0x7f722c068670 0x7f722c068ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 --2- 172.21.15.123:0/1243497542 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0c8b60 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 -- 172.21.15.123:0/1243497542 >> 172.21.15.123:0/1243497542 conn(0x7f722c0c9ca0 msgr2=0x7f722c06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:07.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 -- 172.21.15.123:0/1243497542 shutdown_connections 2023-10-06T04:32:07.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.195+0000 7f72311fc700 1 -- 172.21.15.123:0/1243497542 wait complete. 2023-10-06T04:32:07.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 Processor -- start 2023-10-06T04:32:07.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 -- start start 2023-10-06T04:32:07.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c068670 0x7f722c158130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f722c0c8340 0x7f722c0c60d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0a98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0ad100 con 0x7f722c0c8710 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f722c0adf20 con 0x7f722c068670 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.197+0000 7f72311fc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f722c158670 con 0x7f722c0c8340 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0a98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0a98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42282/0 (socket says 172.21.15.123:42282) 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 -- 172.21.15.123:0/725870806 learned_addr learned my addr 172.21.15.123:0/725870806 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722ad9d700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c068670 0x7f722c158130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722a59c700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f722c0c8340 0x7f722c0c60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 -- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f722c0c8340 msgr2=0x7f722c0c60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f722c0c8340 0x7f722c0c60d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 -- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c068670 msgr2=0x7f722c158130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c068670 0x7f722c158130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 -- 172.21.15.123:0/725870806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009420 con 0x7f722c0c8710 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722ad9d700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c068670 0x7f722c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722a59c700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f722c0c8340 0x7f722c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f722b59e700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0a98b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f722400d180 tx=0x7f7224009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f7212ffd700 1 -- 172.21.15.123:0/725870806 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f722400de20 con 0x7f722c0c8710 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f7212ffd700 1 -- 172.21.15.123:0/725870806 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f722400f040 con 0x7f722c0c8710 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f722c0ae3b0 con 0x7f722c0c8710 2023-10-06T04:32:07.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f7212ffd700 1 -- 172.21.15.123:0/725870806 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224015690 con 0x7f722c0c8710 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.198+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722c0ad670 con 0x7f722c0c8710 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.199+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722c0ca0a0 con 0x7f722c0c8710 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.200+0000 7f7212ffd700 1 -- 172.21.15.123:0/725870806 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f722401e3b0 con 0x7f722c0c8710 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.200+0000 7f7212ffd700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7218066f30 0x7f72180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.200+0000 7f7212ffd700 1 -- 172.21.15.123:0/725870806 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7224084ce0 con 0x7f722c0c8710 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.200+0000 7f722ad9d700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7218066f30 0x7f72180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.200+0000 7f722ad9d700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7218066f30 0x7f72180693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7220001940 tx=0x7f7220009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:07.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.205+0000 7f7212ffd700 1 -- 172.21.15.123:0/725870806 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7224051df0 con 0x7f722c0c8710 2023-10-06T04:32:07.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.389+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f722c0aa6f0 con 0x7f722c0c8710 2023-10-06T04:32:07.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.389+0000 7f7212ffd700 1 -- 172.21.15.123:0/725870806 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7f722401a0a0 con 0x7f722c0c8710 2023-10-06T04:32:07.392 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:32:07.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7218066f30 msgr2=0x7f72180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7218066f30 0x7f72180693e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7220001940 tx=0x7f7220009040 comp rx=0 tx=0).stop 2023-10-06T04:32:07.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 msgr2=0x7f722c0a98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0a98b0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f722400d180 tx=0x7f7224009a50 comp rx=0 tx=0).stop 2023-10-06T04:32:07.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 shutdown_connections 2023-10-06T04:32:07.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7218066f30 0x7f72180693e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722c0c8710 0x7f722c0a98b0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f722c068670 0x7f722c158130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 --2- 172.21.15.123:0/725870806 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f722c0c8340 0x7f722c0c60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 >> 172.21.15.123:0/725870806 conn(0x7f722c0c9ca0 msgr2=0x7f722c06a7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:07.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 shutdown_connections 2023-10-06T04:32:07.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.391+0000 7f72311fc700 1 -- 172.21.15.123:0/725870806 wait complete. 2023-10-06T04:32:07.414 INFO:tasks.cephfs_test_runner:Starting test: test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) 2023-10-06T04:32:07.415 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2023-10-06T04:32:07.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.529+0000 7f7a52ff0700 1 Processor -- start 2023-10-06T04:32:07.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.530+0000 7f7a52ff0700 1 -- start start 2023-10-06T04:32:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.530+0000 7f7a52ff0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7a4c1560c0 0x7f7a4c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.530+0000 7f7a52ff0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 0x7f7a4c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.530+0000 7f7a52ff0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a4c0c8580 con 0x7f7a4c06f080 2023-10-06T04:32:07.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.530+0000 7f7a52ff0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c8860 con 0x7f7a4c1560c0 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.530+0000 7f7a52ff0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c8b40 con 0x7f7a4c06e960 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.530+0000 7f7a4bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f7a4c06f080 0x7f7a4c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53230/0 (socket says 172.21.15.123:53230) 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4bfff700 1 -- 172.21.15.123:0/2617431794 learned_addr learned my addr 172.21.15.123:0/2617431794 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a50d8c700 1 --2- 172.21.15.123:0/2617431794 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c1560c0 0x7f7a4c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a5158d700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 0x7f7a4c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2956559318 0 0) 0x7f7a4c0c8580 con 0x7f7a4c06f080 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7a3c003570 con 0x7f7a4c06f080 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3769308978 0 0) 0x7f7a3c003570 con 0x7f7a4c06f080 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 msgr2=0x7f7a4c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 0x7f7a4c0ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c1560c0 msgr2=0x7f7a4c0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 --2- 172.21.15.123:0/2617431794 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c1560c0 0x7f7a4c0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c0c8e20 con 0x7f7a4c06f080 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f7a38001470 con 0x7f7a4c06f080 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 >> v1:172.21.15.123:6789/0 conn(0x7f7a4c06f080 legacy=0x7f7a4c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 0x7f7a3c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3c003fe0 0x7f7a3c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c004d50 0x7f7a3c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c8860 con 0x7f7a3c003fe0 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c0c8b40 con 0x7f7a3c004d50 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a3c00ba90 con 0x7f7a4c06e960 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a50d8c700 1 --2- 172.21.15.123:0/2617431794 >> v2:172.21.15.177:3300/0 conn(0x7f7a4c1560c0 0x7f7a4c0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:07.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4bfff700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3c003fe0 0x7f7a3c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a5158d700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c004d50 0x7f7a3c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a50d8c700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 0x7f7a3c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4bfff700 1 -- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 msgr2=0x7f7a3c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4bfff700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 0x7f7a3c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4bfff700 1 -- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c004d50 msgr2=0x7f7a3c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.531+0000 7f7a4bfff700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c004d50 0x7f7a3c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a4bfff700 1 -- 172.21.15.123:0/2617431794 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a3c003570 con 0x7f7a3c003fe0 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a50d8c700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c06e960 0x7f7a3c003a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a4bfff700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3c003fe0 0x7f7a3c004810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007d00 tx=0x7f7a38001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=febb2735672d8d64 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800cc00 con 0x7f7a3c003fe0 2023-10-06T04:32:07.539 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a3800cd80 con 0x7f7a3c003fe0 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a4b7fe700 1 -- 172.21.15.123:0/2617431794 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38005b70 con 0x7f7a3c003fe0 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a52ff0700 1 -- 172.21.15.123:0/2617431794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3c003fe0 msgr2=0x7f7a3c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a52ff0700 1 --2- 172.21.15.123:0/2617431794 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a3c003fe0 0x7f7a3c004810 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f7a38007d00 tx=0x7f7a38001e90 comp rx=0 tx=0).stop 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a5158d700 1 -- 172.21.15.123:0/2617431794 reap_dead start 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a52ff0700 1 -- 172.21.15.123:0/2617431794 shutdown_connections 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a52ff0700 1 -- 172.21.15.123:0/2617431794 >> 172.21.15.123:0/2617431794 conn(0x7f7a4c0cda80 msgr2=0x7f7a4c0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a52ff0700 1 -- 172.21.15.123:0/2617431794 shutdown_connections 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.532+0000 7f7a52ff0700 1 -- 172.21.15.123:0/2617431794 wait complete. 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.533+0000 7f7a52ff0700 1 Processor -- start 2023-10-06T04:32:07.540 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a52ff0700 1 -- start start 2023-10-06T04:32:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a52ff0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c003fe0 0x7f7a4c169f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a52ff0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4c1544c0 0x7f7a4c16caf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a52ff0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c16d030 0x7f7a4c0ae930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a52ff0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c16a4b0 con 0x7f7a4c1544c0 2023-10-06T04:32:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a52ff0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c1694f0 con 0x7f7a3c003fe0 2023-10-06T04:32:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a52ff0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a4c16d520 con 0x7f7a4c16d030 2023-10-06T04:32:07.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a4bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4c1544c0 0x7f7a4c16caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a4bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4c1544c0 0x7f7a4c16caf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42294/0 (socket says 172.21.15.123:42294) 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a4bfff700 1 -- 172.21.15.123:0/590101331 learned_addr learned my addr 172.21.15.123:0/590101331 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a50d8c700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c003fe0 0x7f7a4c169f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.534+0000 7f7a5158d700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c16d030 0x7f7a4c0ae930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a4bfff700 1 -- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c16d030 msgr2=0x7f7a4c0ae930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a4bfff700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c16d030 0x7f7a4c0ae930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a4bfff700 1 -- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c003fe0 msgr2=0x7f7a4c169f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a4bfff700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c003fe0 0x7f7a4c169f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a4bfff700 1 -- 172.21.15.123:0/590101331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a4c16d800 con 0x7f7a4c1544c0 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a4bfff700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4c1544c0 0x7f7a4c16caf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001ed0 tx=0x7f7a38017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:07.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a48ff9700 1 -- 172.21.15.123:0/590101331 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a3800cc00 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a4c169770 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a48ff9700 1 -- 172.21.15.123:0/590101331 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a38001430 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a48ff9700 1 -- 172.21.15.123:0/590101331 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a38007530 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.535+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a4c0abc30 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.536+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a4c16d9f0 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.536+0000 7f7a48ff9700 1 -- 172.21.15.123:0/590101331 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7a38011470 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.537+0000 7f7a48ff9700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.537+0000 7f7a48ff9700 1 -- 172.21.15.123:0/590101331 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f7a38085dd0 con 0x7f7a4c1544c0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.537+0000 7f7a50d8c700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.537+0000 7f7a50d8c700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a40014350 tx=0x7f7a40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:07.545 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.541+0000 7f7a48ff9700 1 -- 172.21.15.123:0/590101331 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7a4c16d9f0 con 0x7f7a4c1544c0 2023-10-06T04:32:07.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:07.796+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) v1 -- 0x7f7a4c16d9f0 con 0x7f7a4c1544c0 2023-10-06T04:32:08.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.366+0000 7f7a48ff9700 1 -- 172.21.15.123:0/590101331 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v54) v1 ==== 150+0+0 (secure 0 0 0) 0x7f7a4c16d9f0 con 0x7f7a4c1544c0 2023-10-06T04:32:08.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.368+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7a30066f30 msgr2=0x7f7a300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.368+0000 7f7a52ff0700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7a30066f30 0x7f7a300693e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f7a40014350 tx=0x7f7a40008040 comp rx=0 tx=0).stop 2023-10-06T04:32:08.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.368+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4c1544c0 msgr2=0x7f7a4c16caf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.368+0000 7f7a52ff0700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4c1544c0 0x7f7a4c16caf0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f7a38001ed0 tx=0x7f7a38017000 comp rx=0 tx=0).stop 2023-10-06T04:32:08.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 shutdown_connections 2023-10-06T04:32:08.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7a30066f30 0x7f7a300693e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f7a4c16d030 0x7f7a4c0ae930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7a3c003fe0 0x7f7a4c169f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 --2- 172.21.15.123:0/590101331 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7a4c1544c0 0x7f7a4c16caf0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 >> 172.21.15.123:0/590101331 conn(0x7f7a4c0cda80 msgr2=0x7f7a4c155340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:08.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 shutdown_connections 2023-10-06T04:32:08.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.369+0000 7f7a52ff0700 1 -- 172.21.15.123:0/590101331 wait complete. 2023-10-06T04:32:08.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-10-06T04:32:08.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.509+0000 7f5fc7b96700 1 Processor -- start 2023-10-06T04:32:08.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.510+0000 7f5fc7b96700 1 -- start start 2023-10-06T04:32:08.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc7b96700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 0x7f5fc00a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc7b96700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5fc00c8710 0x7f5fc00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc7b96700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fc0069aa0 con 0x7f5fc0068670 2023-10-06T04:32:08.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc7b96700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fc006a340 con 0x7f5fc00c8710 2023-10-06T04:32:08.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc7b96700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00aa6f0 con 0x7f5fc00c8340 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc6133700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5fc0068670 0x7f5fc00c6d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53242/0 (socket says 172.21.15.123:53242) 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc6133700 1 -- 172.21.15.123:0/2258269295 learned_addr learned my addr 172.21.15.123:0/2258269295 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc5131700 1 --2- 172.21.15.123:0/2258269295 >> v2:172.21.15.177:3300/0 conn(0x7f5fc00c8710 0x7f5fc00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc5932700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 0x7f5fc00a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2562073593 0 0) 0x7f5fc0069aa0 con 0x7f5fc0068670 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fac003570 con 0x7f5fc0068670 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3811106647 0 0) 0x7f5fac003570 con 0x7f5fc0068670 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 msgr2=0x7f5fc00a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:08.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 0x7f5fc00a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 >> v2:172.21.15.177:3300/0 conn(0x7f5fc00c8710 msgr2=0x7f5fc00c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 --2- 172.21.15.123:0/2258269295 >> v2:172.21.15.177:3300/0 conn(0x7f5fc00c8710 0x7f5fc00c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc0069cf0 con 0x7f5fc0068670 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f5fb80017f0 con 0x7f5fc0068670 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 >> v1:172.21.15.123:6789/0 conn(0x7f5fc0068670 legacy=0x7f5fc00c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac003a60 0x7f5fac003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 0x7f5fac004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac004d50 0x7f5fac0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc006a340 con 0x7f5fac003a60 2023-10-06T04:32:08.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00aa6f0 con 0x7f5fac004d50 2023-10-06T04:32:08.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fac00ba60 con 0x7f5fc00c8340 2023-10-06T04:32:08.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc5932700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac003a60 0x7f5fac003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.511+0000 7f5fc5131700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 0x7f5fac004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc6133700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac004d50 0x7f5fac0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc5932700 1 -- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 msgr2=0x7f5fac004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:08.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc5932700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00c8340 0x7f5fac004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc5932700 1 -- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac004d50 msgr2=0x7f5fac0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc5932700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac004d50 0x7f5fac0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc5932700 1 -- 172.21.15.123:0/2258269295 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac003570 con 0x7f5fac003a60 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc5932700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac003a60 0x7f5fac003e70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009c00 tx=0x7f5fb40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e43502f717dafc9e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb400e6b0 con 0x7f5fac003a60 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb400e810 con 0x7f5fac003a60 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc4930700 1 -- 172.21.15.123:0/2258269295 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb400ebd0 con 0x7f5fac003a60 2023-10-06T04:32:08.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc7b96700 1 -- 172.21.15.123:0/2258269295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac003a60 msgr2=0x7f5fac003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc7b96700 1 --2- 172.21.15.123:0/2258269295 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fac003a60 0x7f5fac003e70 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009c00 tx=0x7f5fb40091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.512+0000 7f5fc6133700 1 -- 172.21.15.123:0/2258269295 reap_dead start 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.513+0000 7f5fc7b96700 1 -- 172.21.15.123:0/2258269295 shutdown_connections 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.513+0000 7f5fc7b96700 1 -- 172.21.15.123:0/2258269295 >> 172.21.15.123:0/2258269295 conn(0x7f5fc00c9ca0 msgr2=0x7f5fc006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.513+0000 7f5fc7b96700 1 -- 172.21.15.123:0/2258269295 shutdown_connections 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.513+0000 7f5fc7b96700 1 -- 172.21.15.123:0/2258269295 wait complete. 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 Processor -- start 2023-10-06T04:32:08.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 -- start start 2023-10-06T04:32:08.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac003a60 0x7f5fc00c4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc0155d60 0x7f5fc01698b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00ac6e0 0x7f5fc00cb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.523 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00ac250 con 0x7f5fc0155d60 2023-10-06T04:32:08.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc00c5100 con 0x7f5fac003a60 2023-10-06T04:32:08.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.514+0000 7f5fc7b96700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fc016a200 con 0x7f5fc00ac6e0 2023-10-06T04:32:08.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc0155d60 0x7f5fc01698b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc0155d60 0x7f5fc01698b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42308/0 (socket says 172.21.15.123:42308) 2023-10-06T04:32:08.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 -- 172.21.15.123:0/170988478 learned_addr learned my addr 172.21.15.123:0/170988478 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:08.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5932700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac003a60 0x7f5fc00c4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc6133700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00ac6e0 0x7f5fc00cb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 -- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00ac6e0 msgr2=0x7f5fc00cb0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00ac6e0 0x7f5fc00cb0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 -- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac003a60 msgr2=0x7f5fc00c4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac003a60 0x7f5fc00c4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 -- 172.21.15.123:0/170988478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc016a4e0 con 0x7f5fc0155d60 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc6133700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00ac6e0 0x7f5fc00cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5932700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac003a60 0x7f5fc00c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:08.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc5131700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc0155d60 0x7f5fc01698b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc011d40 tx=0x7f5fbc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fb1ffb700 1 -- 172.21.15.123:0/170988478 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc00fac0 con 0x7f5fc0155d60 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fb1ffb700 1 -- 172.21.15.123:0/170988478 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fbc00fc20 con 0x7f5fc0155d60 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fc00abee0 con 0x7f5fc0155d60 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fb1ffb700 1 -- 172.21.15.123:0/170988478 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc014440 con 0x7f5fc0155d60 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.515+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fc0169df0 con 0x7f5fc0155d60 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.516+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fc0069fd0 con 0x7f5fc0155d60 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.517+0000 7f5fb1ffb700 1 -- 172.21.15.123:0/170988478 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fbc014a80 con 0x7f5fc0155d60 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.517+0000 7f5fb1ffb700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5fa4066f30 0x7f5fa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:08.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.517+0000 7f5fb1ffb700 1 -- 172.21.15.123:0/170988478 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f5fbc018030 con 0x7f5fc0155d60 2023-10-06T04:32:08.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.517+0000 7f5fc5932700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5fa4066f30 0x7f5fa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:08.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.517+0000 7f5fc5932700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5fa4066f30 0x7f5fa40693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40091e0 tx=0x7f5fb401a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:08.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.521+0000 7f5fb1ffb700 1 -- 172.21.15.123:0/170988478 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fbc052460 con 0x7f5fc0155d60 2023-10-06T04:32:08.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.703+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f5fc004bb80 con 0x7f5fc0155d60 2023-10-06T04:32:08.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.706+0000 7f5fb1ffb700 1 -- 172.21.15.123:0/170988478 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5fbc055a80 con 0x7f5fc0155d60 2023-10-06T04:32:08.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5fa4066f30 msgr2=0x7f5fa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5fa4066f30 0x7f5fa40693e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f5fb40091e0 tx=0x7f5fb401a000 comp rx=0 tx=0).stop 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc0155d60 msgr2=0x7f5fc01698b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc0155d60 0x7f5fc01698b0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc011d40 tx=0x7f5fbc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 shutdown_connections 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5fa4066f30 0x7f5fa40693e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fc00ac6e0 0x7f5fc00cb0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fac003a60 0x7f5fc00c4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 --2- 172.21.15.123:0/170988478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fc0155d60 0x7f5fc01698b0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 >> 172.21.15.123:0/170988478 conn(0x7f5fc00c9ca0 msgr2=0x7f5fc0158b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 shutdown_connections 2023-10-06T04:32:08.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:08.708+0000 7f5fc7b96700 1 -- 172.21.15.123:0/170988478 wait complete. 2023-10-06T04:32:08.721 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:32:08.722 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:32:08.759 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:32:08.812 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:32:08.812 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2023-10-06T04:32:08.812 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:32:08.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:08.855 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:32:08.856 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:32:08.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:08.856 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:32:08.939 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi177.front.sepia.ceph.com... 2023-10-06T04:32:08.939 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:32:08.939 DEBUG:teuthology.orchestra.run.smithi177:> sudo fusermount -u /home/ubuntu/cephtest/mnt.1 2023-10-06T04:32:08.981 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:32:08.981 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:32:08.982 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:08.982 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:32:09.055 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2023-10-06T04:32:09.055 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:32:09.055 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.2 2023-10-06T04:32:09.092 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:32:09.092 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:32:09.093 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:09.093 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:32:09.151 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi177.front.sepia.ceph.com... 2023-10-06T04:32:09.153 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:32:09.153 DEBUG:teuthology.orchestra.run.smithi177:> sudo fusermount -u /home/ubuntu/cephtest/mnt.3 2023-10-06T04:32:09.192 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:32:09.193 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:32:09.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:09.193 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:32:09.253 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-10-06T04:32:09.253 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:09.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.370+0000 7f6afeae5700 1 Processor -- start 2023-10-06T04:32:09.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afeae5700 1 -- start start 2023-10-06T04:32:09.375 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afeae5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 0x7f6af80c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afeae5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6af806f080 0x7f6af806e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afeae5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6af80c8580 con 0x7f6af8156490 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afeae5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6af80c8860 con 0x7f6af806f080 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afeae5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c8b40 con 0x7f6af81560c0 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afd082700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6af8156490 0x7f6af806a370 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53258/0 (socket says 172.21.15.123:53258) 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.371+0000 7f6afd082700 1 -- 172.21.15.123:0/1763597777 learned_addr learned my addr 172.21.15.123:0/1763597777 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 125106001 0 0) 0x7f6af80c8580 con 0x7f6af8156490 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6ae8003570 con 0x7f6af8156490 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af7fff700 1 --2- 172.21.15.123:0/1763597777 >> v2:172.21.15.177:3300/0 conn(0x7f6af806f080 0x7f6af806e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afc881700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 0x7f6af80c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1786523185 0 0) 0x7f6ae8003570 con 0x7f6af8156490 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 msgr2=0x7f6af80c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 0x7f6af80c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 >> v2:172.21.15.177:3300/0 conn(0x7f6af806f080 msgr2=0x7f6af806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 --2- 172.21.15.123:0/1763597777 >> v2:172.21.15.177:3300/0 conn(0x7f6af806f080 0x7f6af806e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af80c8e20 con 0x7f6af8156490 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f6af00017f0 con 0x7f6af8156490 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 >> v1:172.21.15.123:6789/0 conn(0x7f6af8156490 legacy=0x7f6af806a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8003a60 0x7f6ae8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 0x7f6ae8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8004d50 0x7f6ae80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c8860 con 0x7f6ae8004d50 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80c8b40 con 0x7f6ae8003a60 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ae800ba60 con 0x7f6af81560c0 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afd082700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8004d50 0x7f6ae80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afc881700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8003a60 0x7f6ae8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6af7fff700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 0x7f6ae8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afd082700 1 -- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 msgr2=0x7f6ae8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afd082700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af81560c0 0x7f6ae8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afd082700 1 -- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8003a60 msgr2=0x7f6ae8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afd082700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8003a60 0x7f6ae8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.372+0000 7f6afd082700 1 -- 172.21.15.123:0/1763597777 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ae8003570 con 0x7f6ae8004d50 2023-10-06T04:32:09.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afd082700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8004d50 0x7f6ae80051c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6af000a970 tx=0x7f6af806f4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80c8bea3f3971567 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00070d0 con 0x7f6ae8004d50 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6af0007230 con 0x7f6ae8004d50 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6af77fe700 1 -- 172.21.15.123:0/1763597777 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6af00075f0 con 0x7f6ae8004d50 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afeae5700 1 -- 172.21.15.123:0/1763597777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8004d50 msgr2=0x7f6ae80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afeae5700 1 --2- 172.21.15.123:0/1763597777 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ae8004d50 0x7f6ae80051c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f6af000a970 tx=0x7f6af806f4f0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afd082700 1 -- 172.21.15.123:0/1763597777 reap_dead start 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afeae5700 1 -- 172.21.15.123:0/1763597777 shutdown_connections 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afeae5700 1 -- 172.21.15.123:0/1763597777 >> 172.21.15.123:0/1763597777 conn(0x7f6af80cda80 msgr2=0x7f6af80c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afeae5700 1 -- 172.21.15.123:0/1763597777 shutdown_connections 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.373+0000 7f6afeae5700 1 -- 172.21.15.123:0/1763597777 wait complete. 2023-10-06T04:32:09.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.374+0000 7f6afeae5700 1 Processor -- start 2023-10-06T04:32:09.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.375+0000 7f6afeae5700 1 -- start start 2023-10-06T04:32:09.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.375+0000 7f6afeae5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8004d50 0x7f6af8159e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.375+0000 7f6afeae5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8153e00 0x7f6af815b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.375+0000 7f6afeae5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af80cbd30 0x7f6af80cc180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.375+0000 7f6afeae5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af80cc6c0 con 0x7f6af8153e00 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.375+0000 7f6afeae5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6af8156c20 con 0x7f6ae8004d50 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.375+0000 7f6afeae5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6af8166210 con 0x7f6af80cbd30 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8153e00 0x7f6af815b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8153e00 0x7f6af815b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42324/0 (socket says 172.21.15.123:42324) 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 -- 172.21.15.123:0/2675149152 learned_addr learned my addr 172.21.15.123:0/2675149152 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6afc881700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8004d50 0x7f6af8159e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6afd082700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af80cbd30 0x7f6af80cc180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 -- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af80cbd30 msgr2=0x7f6af80cc180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af80cbd30 0x7f6af80cc180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 -- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8004d50 msgr2=0x7f6af8159e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8004d50 0x7f6af8159e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 -- 172.21.15.123:0/2675149152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6af81664f0 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af7fff700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8153e00 0x7f6af815b8f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011c20 tx=0x7f6ae40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af4ff9700 1 -- 172.21.15.123:0/2675149152 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae4016070 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af4ff9700 1 -- 172.21.15.123:0/2675149152 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ae400e930 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6af81667d0 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6af4ff9700 1 -- 172.21.15.123:0/2675149152 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ae400ecb0 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.376+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6af8166d90 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.377+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6af80cde80 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.378+0000 7f6af4ff9700 1 -- 172.21.15.123:0/2675149152 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ae401b630 con 0x7f6af8153e00 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.378+0000 7f6af4ff9700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6adc066fe0 0x7f6adc069490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.378+0000 7f6afc881700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6adc066fe0 0x7f6adc069490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.378+0000 7f6af4ff9700 1 -- 172.21.15.123:0/2675149152 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ae4086c40 con 0x7f6af8153e00 2023-10-06T04:32:09.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.378+0000 7f6afc881700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6adc066fe0 0x7f6adc069490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6af80cca70 tx=0x7f6aec0091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:09.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.382+0000 7f6af4ff9700 1 -- 172.21.15.123:0/2675149152 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ae4053d30 con 0x7f6af8153e00 2023-10-06T04:32:09.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.608+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6af804bb80 con 0x7f6af8153e00 2023-10-06T04:32:09.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.608+0000 7f6af4ff9700 1 -- 172.21.15.123:0/2675149152 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f6ae4057350 con 0x7f6af8153e00 2023-10-06T04:32:09.610 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:09.610 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:32:09.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6adc066fe0 msgr2=0x7f6adc069490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6adc066fe0 0x7f6adc069490 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f6af80cca70 tx=0x7f6aec0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8153e00 msgr2=0x7f6af815b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8153e00 0x7f6af815b8f0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ae4011c20 tx=0x7f6ae40091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 shutdown_connections 2023-10-06T04:32:09.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6adc066fe0 0x7f6adc069490 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6af80cbd30 0x7f6af80cc180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ae8004d50 0x7f6af8159e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 --2- 172.21.15.123:0/2675149152 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6af8153e00 0x7f6af815b8f0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 >> 172.21.15.123:0/2675149152 conn(0x7f6af80cda80 msgr2=0x7f6af80cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:09.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 shutdown_connections 2023-10-06T04:32:09.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.610+0000 7f6afeae5700 1 -- 172.21.15.123:0/2675149152 wait complete. 2023-10-06T04:32:09.615 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:32:09.624 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:09.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.740+0000 7f177d3c0700 1 Processor -- start 2023-10-06T04:32:09.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.740+0000 7f177d3c0700 1 -- start start 2023-10-06T04:32:09.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f177d3c0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17780aae60 0x7f17780aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f177d3c0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17780abe30 con 0x7f17780ab230 2023-10-06T04:32:09.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f177d3c0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17780ac9b0 con 0x7f17780ace40 2023-10-06T04:32:09.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f177d3c0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17780ac080 con 0x7f17780aae60 2023-10-06T04:32:09.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f1776ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17780aae60 0x7f17780aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f176ffff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f17780ab230 0x7f17780afbf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53264/0 (socket says 172.21.15.123:53264) 2023-10-06T04:32:09.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f176ffff700 1 -- 172.21.15.123:0/3959007010 learned_addr learned my addr 172.21.15.123:0/3959007010 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:09.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f1776ffd700 1 -- 172.21.15.123:0/3959007010 >> v1:172.21.15.177:6789/0 conn(0x7f17780ace40 legacy=0x7f1778068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:09.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f1776ffd700 1 -- 172.21.15.123:0/3959007010 >> v1:172.21.15.123:6789/0 conn(0x7f17780ab230 legacy=0x7f17780afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f1776ffd700 1 -- 172.21.15.123:0/3959007010 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17780ac360 con 0x7f17780aae60 2023-10-06T04:32:09.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f17767fc700 1 -- 172.21.15.123:0/3959007010 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 346256012 0 0) 0x7f17780abe30 con 0x7f17780ab230 2023-10-06T04:32:09.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f1776ffd700 1 --2- 172.21.15.123:0/3959007010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780aae60 0x7f17780aec10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f17680098e0 tx=0x7f17680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c83fbc78ffe1c3df server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:09.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f17767fc700 1 -- 172.21.15.123:0/3959007010 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1768009e80 con 0x7f17780aae60 2023-10-06T04:32:09.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f17767fc700 1 -- 172.21.15.123:0/3959007010 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f176800b820 con 0x7f17780aae60 2023-10-06T04:32:09.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.741+0000 7f17767fc700 1 -- 172.21.15.123:0/3959007010 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f176800bba0 con 0x7f17780aae60 2023-10-06T04:32:09.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.742+0000 7f177d3c0700 1 -- 172.21.15.123:0/3959007010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780aae60 msgr2=0x7f17780aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.742+0000 7f177d3c0700 1 --2- 172.21.15.123:0/3959007010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780aae60 0x7f17780aec10 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f17680098e0 tx=0x7f17680091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.742+0000 7f177d3c0700 1 -- 172.21.15.123:0/3959007010 shutdown_connections 2023-10-06T04:32:09.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.742+0000 7f177d3c0700 1 --2- 172.21.15.123:0/3959007010 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780aae60 0x7f17780aec10 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.742+0000 7f177d3c0700 1 -- 172.21.15.123:0/3959007010 >> 172.21.15.123:0/3959007010 conn(0x7f1778097410 msgr2=0x7f1778067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:09.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.742+0000 7f177d3c0700 1 -- 172.21.15.123:0/3959007010 shutdown_connections 2023-10-06T04:32:09.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.742+0000 7f177d3c0700 1 -- 172.21.15.123:0/3959007010 wait complete. 2023-10-06T04:32:09.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.743+0000 7f177d3c0700 1 Processor -- start 2023-10-06T04:32:09.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.743+0000 7f177d3c0700 1 -- start start 2023-10-06T04:32:09.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f177d3c0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17780aae60 0x7f1778146af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f177d3c0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17780ab230 0x7f1778157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f177d3c0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780ace40 0x7f1778158140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f177d3c0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177815a770 con 0x7f17780ace40 2023-10-06T04:32:09.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f177d3c0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f177815ac50 con 0x7f17780aae60 2023-10-06T04:32:09.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f177d3c0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f177809a3f0 con 0x7f17780ab230 2023-10-06T04:32:09.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780ace40 0x7f1778158140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780ace40 0x7f1778158140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42340/0 (socket says 172.21.15.123:42340) 2023-10-06T04:32:09.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 -- 172.21.15.123:0/2619726815 learned_addr learned my addr 172.21.15.123:0/2619726815 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:09.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f1776ffd700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17780aae60 0x7f1778146af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 -- 172.21.15.123:0/2619726815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17780ab230 msgr2=0x7f1778157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17780ab230 0x7f1778157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 -- 172.21.15.123:0/2619726815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17780aae60 msgr2=0x7f1778146af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17780aae60 0x7f1778146af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 -- 172.21.15.123:0/2619726815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1768009420 con 0x7f17780ace40 2023-10-06T04:32:09.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f17777fe700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780ace40 0x7f1778158140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f177000bbb0 tx=0x7f1770009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:09.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.744+0000 7f176effd700 1 -- 172.21.15.123:0/2619726815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17700092e0 con 0x7f17780ace40 2023-10-06T04:32:09.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.745+0000 7f176effd700 1 -- 172.21.15.123:0/2619726815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f177000c7a0 con 0x7f17780ace40 2023-10-06T04:32:09.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.745+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1778159970 con 0x7f17780ace40 2023-10-06T04:32:09.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.745+0000 7f176effd700 1 -- 172.21.15.123:0/2619726815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f177000cb40 con 0x7f17780ace40 2023-10-06T04:32:09.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.745+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f177809a080 con 0x7f17780ace40 2023-10-06T04:32:09.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.745+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f177806d550 con 0x7f17780ace40 2023-10-06T04:32:09.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.746+0000 7f176effd700 1 -- 172.21.15.123:0/2619726815 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1770012070 con 0x7f17780ace40 2023-10-06T04:32:09.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.746+0000 7f176effd700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1758066f30 0x7f17580693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:09.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.746+0000 7f176effd700 1 -- 172.21.15.123:0/2619726815 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f17700842f0 con 0x7f17780ace40 2023-10-06T04:32:09.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.746+0000 7f1776ffd700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1758066f30 0x7f17580693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:09.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.747+0000 7f1776ffd700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1758066f30 0x7f17580693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1768006d80 tx=0x7f1768016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:09.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.750+0000 7f176effd700 1 -- 172.21.15.123:0/2619726815 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1770051400 con 0x7f17780ace40 2023-10-06T04:32:09.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.933+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f177806f740 con 0x7f17780ace40 2023-10-06T04:32:09.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.934+0000 7f176effd700 1 -- 172.21.15.123:0/2619726815 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f1770054a20 con 0x7f17780ace40 2023-10-06T04:32:09.936 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:09.936 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:09.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1758066f30 msgr2=0x7f17580693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1758066f30 0x7f17580693e0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f1768006d80 tx=0x7f1768016000 comp rx=0 tx=0).stop 2023-10-06T04:32:09.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780ace40 msgr2=0x7f1778158140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:09.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780ace40 0x7f1778158140 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f177000bbb0 tx=0x7f1770009840 comp rx=0 tx=0).stop 2023-10-06T04:32:09.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 shutdown_connections 2023-10-06T04:32:09.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1758066f30 0x7f17580693e0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17780ace40 0x7f1778158140 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f17780aae60 0x7f1778146af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 --2- 172.21.15.123:0/2619726815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f17780ab230 0x7f1778157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:09.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 >> 172.21.15.123:0/2619726815 conn(0x7f1778097410 msgr2=0x7f17780b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:09.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 shutdown_connections 2023-10-06T04:32:09.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:09.936+0000 7f177d3c0700 1 -- 172.21.15.123:0/2619726815 wait complete. 2023-10-06T04:32:09.950 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 3, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:31:32.045143+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '22', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 4, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:31:33.390421+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '23', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:32:09.951 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:10.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.069+0000 7f6ac123d700 1 Processor -- start 2023-10-06T04:32:10.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.069+0000 7f6ac123d700 1 -- start start 2023-10-06T04:32:10.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6ac123d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc157080 0x7f6abc0abf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6ac123d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6abc068d80 0x7f6abc0c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6ac123d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6abc0c7810 con 0x7f6abc0ac470 2023-10-06T04:32:10.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6ac123d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6abc0ac9f0 con 0x7f6abc068d80 2023-10-06T04:32:10.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6ac123d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0c6ee0 con 0x7f6abc157080 2023-10-06T04:32:10.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6aba59c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f6abc0ac470 0x7f6abc068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:53276/0 (socket says 172.21.15.123:53276) 2023-10-06T04:32:10.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6abb59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6abc068d80 0x7f6abc0c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6aba59c700 1 -- 172.21.15.123:0/2195767633 learned_addr learned my addr 172.21.15.123:0/2195767633 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:10.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.070+0000 7f6abad9d700 1 --2- 172.21.15.123:0/2195767633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc157080 0x7f6abc0abf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6abb59e700 1 -- 172.21.15.123:0/2195767633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc157080 msgr2=0x7f6abc0abf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:10.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6abb59e700 1 --2- 172.21.15.123:0/2195767633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc157080 0x7f6abc0abf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6abb59e700 1 -- 172.21.15.123:0/2195767633 >> v1:172.21.15.123:6789/0 conn(0x7f6abc0ac470 legacy=0x7f6abc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ab9d9b700 1 -- 172.21.15.123:0/2195767633 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3266016270 0 0) 0x7f6abc0c7810 con 0x7f6abc0ac470 2023-10-06T04:32:10.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6abb59e700 1 -- 172.21.15.123:0/2195767633 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6abc0c71c0 con 0x7f6abc068d80 2023-10-06T04:32:10.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6abad9d700 1 --2- 172.21.15.123:0/2195767633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc157080 0x7f6abc0abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:10.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6abb59e700 1 --2- 172.21.15.123:0/2195767633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc068d80 0x7f6abc0c7ca0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009e90 tx=0x7f6ab4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ab91bab12cf166 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ab9d9b700 1 -- 172.21.15.123:0/2195767633 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab400e070 con 0x7f6abc068d80 2023-10-06T04:32:10.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ab9d9b700 1 -- 172.21.15.123:0/2195767633 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab400bbf0 con 0x7f6abc068d80 2023-10-06T04:32:10.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ab9d9b700 1 -- 172.21.15.123:0/2195767633 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab4012430 con 0x7f6abc068d80 2023-10-06T04:32:10.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 -- 172.21.15.123:0/2195767633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc068d80 msgr2=0x7f6abc0c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2195767633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc068d80 0x7f6abc0c7ca0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4009e90 tx=0x7f6ab4009780 comp rx=0 tx=0).stop 2023-10-06T04:32:10.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 -- 172.21.15.123:0/2195767633 shutdown_connections 2023-10-06T04:32:10.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2195767633 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc068d80 0x7f6abc0c7ca0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.082 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2195767633 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc157080 0x7f6abc0abf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 -- 172.21.15.123:0/2195767633 >> 172.21.15.123:0/2195767633 conn(0x7f6abc0c9ca0 msgr2=0x7f6abc06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:10.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 -- 172.21.15.123:0/2195767633 shutdown_connections 2023-10-06T04:32:10.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.071+0000 7f6ac123d700 1 -- 172.21.15.123:0/2195767633 wait complete. 2023-10-06T04:32:10.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 Processor -- start 2023-10-06T04:32:10.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 -- start start 2023-10-06T04:32:10.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc068d80 0x7f6abc154050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6abc0ac470 0x7f6abc0c50d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc157080 0x7f6abc0c5610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0b08d0 con 0x7f6abc157080 2023-10-06T04:32:10.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0c4a50 con 0x7f6abc0ac470 2023-10-06T04:32:10.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6ac123d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6abc0c5c90 con 0x7f6abc068d80 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6abb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc157080 0x7f6abc0c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6abb59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc157080 0x7f6abc0c5610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42354/0 (socket says 172.21.15.123:42354) 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.073+0000 7f6abb59e700 1 -- 172.21.15.123:0/2001564275 learned_addr learned my addr 172.21.15.123:0/2001564275 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6abad9d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc068d80 0x7f6abc154050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6aba59c700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6abc0ac470 0x7f6abc0c50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6abb59e700 1 -- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc068d80 msgr2=0x7f6abc154050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6abb59e700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc068d80 0x7f6abc154050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6abb59e700 1 -- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6abc0ac470 msgr2=0x7f6abc0c50d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6abb59e700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6abc0ac470 0x7f6abc0c50d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6abb59e700 1 -- 172.21.15.123:0/2001564275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ab4009a20 con 0x7f6abc157080 2023-10-06T04:32:10.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6aba59c700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6abc0ac470 0x7f6abc0c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:10.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6abb59e700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc157080 0x7f6abc0c5610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4013000 tx=0x7f6ab4019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6aa2ffd700 1 -- 172.21.15.123:0/2001564275 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab401d630 con 0x7f6abc157080 2023-10-06T04:32:10.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6aa2ffd700 1 -- 172.21.15.123:0/2001564275 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ab401d790 con 0x7f6abc157080 2023-10-06T04:32:10.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6abc0c5f70 con 0x7f6abc157080 2023-10-06T04:32:10.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6aa2ffd700 1 -- 172.21.15.123:0/2001564275 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ab401db50 con 0x7f6abc157080 2023-10-06T04:32:10.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.074+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6abc0c6530 con 0x7f6abc157080 2023-10-06T04:32:10.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.075+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6abc0b0a40 con 0x7f6abc157080 2023-10-06T04:32:10.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.076+0000 7f6aa2ffd700 1 -- 172.21.15.123:0/2001564275 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ab4024070 con 0x7f6abc157080 2023-10-06T04:32:10.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.076+0000 7f6aa2ffd700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6aa8066f30 0x7f6aa80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.076+0000 7f6aa2ffd700 1 -- 172.21.15.123:0/2001564275 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6ab4086f60 con 0x7f6abc157080 2023-10-06T04:32:10.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.076+0000 7f6abad9d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6aa8066f30 0x7f6aa80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.076+0000 7f6abad9d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6aa8066f30 0x7f6aa80693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0006fd0 tx=0x7f6ab0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.080+0000 7f6aa2ffd700 1 -- 172.21.15.123:0/2001564275 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6abc0b0a40 con 0x7f6abc157080 2023-10-06T04:32:10.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.258+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6abc0b0a10 con 0x7f6abc157080 2023-10-06T04:32:10.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.259+0000 7f6aa2ffd700 1 -- 172.21.15.123:0/2001564275 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v23) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6ab4054070 con 0x7f6abc157080 2023-10-06T04:32:10.260 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6aa8066f30 msgr2=0x7f6aa80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6aa8066f30 0x7f6aa80693e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f6ab0006fd0 tx=0x7f6ab0008040 comp rx=0 tx=0).stop 2023-10-06T04:32:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc157080 msgr2=0x7f6abc0c5610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc157080 0x7f6abc0c5610 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f6ab4013000 tx=0x7f6ab4019000 comp rx=0 tx=0).stop 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 shutdown_connections 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6aa8066f30 0x7f6aa80693e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6abc157080 0x7f6abc0c5610 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6abc068d80 0x7f6abc154050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 --2- 172.21.15.123:0/2001564275 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6abc0ac470 0x7f6abc0c50d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 >> 172.21.15.123:0/2001564275 conn(0x7f6abc0c9ca0 msgr2=0x7f6abc155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 shutdown_connections 2023-10-06T04:32:10.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.261+0000 7f6ac123d700 1 -- 172.21.15.123:0/2001564275 wait complete. 2023-10-06T04:32:10.274 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:10.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.393+0000 7f65b46b5700 1 Processor -- start 2023-10-06T04:32:10.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.393+0000 7f65b46b5700 1 -- start start 2023-10-06T04:32:10.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.394+0000 7f65b46b5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 0x7f65ac0a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.394+0000 7f65b46b5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f65ac068d80 0x7f65ac0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.394+0000 7f65b46b5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65ac069aa0 con 0x7f65ac0c8710 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.394+0000 7f65b46b5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65ac06a340 con 0x7f65ac068d80 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.394+0000 7f65b46b5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0aa6f0 con 0x7f65ac0c8340 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.394+0000 7f65b1c50700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f65ac0c8710 0x7f65ac068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39274/0 (socket says 172.21.15.123:39274) 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.394+0000 7f65b1c50700 1 -- 172.21.15.123:0/1940095668 learned_addr learned my addr 172.21.15.123:0/1940095668 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2c52700 1 --2- 172.21.15.123:0/1940095668 >> v2:172.21.15.177:3300/0 conn(0x7f65ac068d80 0x7f65ac0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2451700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 0x7f65ac0a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1453224058 0 0) 0x7f65ac069aa0 con 0x7f65ac0c8710 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6598003570 con 0x7f65ac0c8710 2023-10-06T04:32:10.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2793108242 0 0) 0x7f6598003570 con 0x7f65ac0c8710 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 msgr2=0x7f65ac0a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 0x7f65ac0a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 >> v2:172.21.15.177:3300/0 conn(0x7f65ac068d80 msgr2=0x7f65ac0c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 --2- 172.21.15.123:0/1940095668 >> v2:172.21.15.177:3300/0 conn(0x7f65ac068d80 0x7f65ac0c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac069cf0 con 0x7f65ac0c8710 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f65a8001470 con 0x7f65ac0c8710 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 >> v1:172.21.15.123:6789/0 conn(0x7f65ac0c8710 legacy=0x7f65ac068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598003a60 0x7f6598003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 0x7f6598004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6598004d50 0x7f65980051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac06a340 con 0x7f6598003a60 2023-10-06T04:32:10.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac0aa6f0 con 0x7f6598004d50 2023-10-06T04:32:10.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f659800ba60 con 0x7f65ac0c8340 2023-10-06T04:32:10.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2451700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598003a60 0x7f6598003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b1c50700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 0x7f6598004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2c52700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6598004d50 0x7f65980051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2451700 1 -- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 msgr2=0x7f6598004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2451700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f65ac0c8340 0x7f6598004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2451700 1 -- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6598004d50 msgr2=0x7f65980051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2451700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6598004d50 0x7f65980051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.395+0000 7f65b2451700 1 -- 172.21.15.123:0/1940095668 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6598003570 con 0x7f6598003a60 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b2451700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598003a60 0x7f6598003e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f659c009c00 tx=0x7f659c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5feef7bb03db400 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00e6b0 con 0x7f6598003a60 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f659c00e810 con 0x7f6598003a60 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b144f700 1 -- 172.21.15.123:0/1940095668 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f659c00ebd0 con 0x7f6598003a60 2023-10-06T04:32:10.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b46b5700 1 -- 172.21.15.123:0/1940095668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598003a60 msgr2=0x7f6598003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b46b5700 1 --2- 172.21.15.123:0/1940095668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6598003a60 0x7f6598003e70 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f659c009c00 tx=0x7f659c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b2c52700 1 -- 172.21.15.123:0/1940095668 reap_dead start 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b46b5700 1 -- 172.21.15.123:0/1940095668 shutdown_connections 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b46b5700 1 -- 172.21.15.123:0/1940095668 >> 172.21.15.123:0/1940095668 conn(0x7f65ac0c9ca0 msgr2=0x7f65ac06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b46b5700 1 -- 172.21.15.123:0/1940095668 shutdown_connections 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.396+0000 7f65b46b5700 1 -- 172.21.15.123:0/1940095668 wait complete. 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.397+0000 7f65b46b5700 1 Processor -- start 2023-10-06T04:32:10.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.397+0000 7f65b46b5700 1 -- start start 2023-10-06T04:32:10.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b46b5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6598003a60 0x7f65ac15bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b46b5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65ac15b490 0x7f65ac15a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b46b5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac0ca490 0x7f65ac157b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b46b5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac15aa30 con 0x7f65ac0ca490 2023-10-06T04:32:10.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b46b5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac15c450 con 0x7f65ac15b490 2023-10-06T04:32:10.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b46b5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65ac15c730 con 0x7f6598003a60 2023-10-06T04:32:10.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b2c52700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac0ca490 0x7f65ac157b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b2c52700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac0ca490 0x7f65ac157b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50418/0 (socket says 172.21.15.123:50418) 2023-10-06T04:32:10.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b2c52700 1 -- 172.21.15.123:0/2092182471 learned_addr learned my addr 172.21.15.123:0/2092182471 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:10.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b1c50700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65ac15b490 0x7f65ac15a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.398+0000 7f65b2451700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6598003a60 0x7f65ac15bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b2c52700 1 -- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6598003a60 msgr2=0x7f65ac15bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b2c52700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6598003a60 0x7f65ac15bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b2c52700 1 -- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65ac15b490 msgr2=0x7f65ac15a4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b2c52700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65ac15b490 0x7f65ac15a4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b2c52700 1 -- 172.21.15.123:0/2092182471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65ac159cf0 con 0x7f65ac0ca490 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b2c52700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac0ca490 0x7f65ac157b10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0af3f0 tx=0x7f65a400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65a27fc700 1 -- 172.21.15.123:0/2092182471 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a400e040 con 0x7f65ac0ca490 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65a27fc700 1 -- 172.21.15.123:0/2092182471 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65a400f040 con 0x7f65ac0ca490 2023-10-06T04:32:10.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65ac159fd0 con 0x7f65ac0ca490 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65a27fc700 1 -- 172.21.15.123:0/2092182471 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65a400fd70 con 0x7f65ac0ca490 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.399+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65ac0ca910 con 0x7f65ac0ca490 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.400+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65ac15a1c0 con 0x7f65ac0ca490 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.400+0000 7f65a27fc700 1 -- 172.21.15.123:0/2092182471 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65a4015640 con 0x7f65ac0ca490 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.401+0000 7f65a27fc700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6590066f30 0x7f65900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.401+0000 7f65a27fc700 1 -- 172.21.15.123:0/2092182471 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f65a4084d20 con 0x7f65ac0ca490 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.401+0000 7f65b2451700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6590066f30 0x7f65900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.401+0000 7f65b2451700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6590066f30 0x7f65900693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f659c0091e0 tx=0x7f659c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.405+0000 7f65a27fc700 1 -- 172.21.15.123:0/2092182471 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65a4051e30 con 0x7f65ac0ca490 2023-10-06T04:32:10.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.584+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f65ac069fd0 con 0x7f65ac0ca490 2023-10-06T04:32:10.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.584+0000 7f65a27fc700 1 -- 172.21.15.123:0/2092182471 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v23) v1 ==== 92+0+10 (secure 0 0 0) 0x7f65a4055450 con 0x7f65ac0ca490 2023-10-06T04:32:10.586 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:10.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6590066f30 msgr2=0x7f65900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6590066f30 0x7f65900693e0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f659c0091e0 tx=0x7f659c01a000 comp rx=0 tx=0).stop 2023-10-06T04:32:10.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac0ca490 msgr2=0x7f65ac157b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac0ca490 0x7f65ac157b10 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f65ac0af3f0 tx=0x7f65a400b040 comp rx=0 tx=0).stop 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 shutdown_connections 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6590066f30 0x7f65900693e0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f65ac0ca490 0x7f65ac157b10 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6598003a60 0x7f65ac15bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 --2- 172.21.15.123:0/2092182471 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f65ac15b490 0x7f65ac15a4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 >> 172.21.15.123:0/2092182471 conn(0x7f65ac0c9ca0 msgr2=0x7f65ac0cbff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 shutdown_connections 2023-10-06T04:32:10.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.587+0000 7f65b46b5700 1 -- 172.21.15.123:0/2092182471 wait complete. 2023-10-06T04:32:10.600 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:32:10.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.718+0000 7f92e66f2700 1 Processor -- start 2023-10-06T04:32:10.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.718+0000 7f92e66f2700 1 -- start start 2023-10-06T04:32:10.722 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.719+0000 7f92e66f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e00cab00 0x7f92e01607a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.719+0000 7f92e66f2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e00c2b80 con 0x7f92e0155a50 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.719+0000 7f92e66f2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e006f4f0 con 0x7f92e0155680 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.719+0000 7f92e66f2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e006e640 con 0x7f92e00cab00 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.719+0000 7f92df7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f92e0155a50 0x7f92e00c1600 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39278/0 (socket says 172.21.15.123:39278) 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.719+0000 7f92df7fe700 1 -- 172.21.15.123:0/2616600941 learned_addr learned my addr 172.21.15.123:0/2616600941 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92e4c8f700 1 --2- 172.21.15.123:0/2616600941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e00cab00 0x7f92e01607a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92deffd700 1 -- 172.21.15.123:0/2616600941 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4190116601 0 0) 0x7f92e00c2b80 con 0x7f92e0155a50 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92deffd700 1 -- 172.21.15.123:0/2616600941 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c8003570 con 0x7f92e0155a50 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92deffd700 1 -- 172.21.15.123:0/2616600941 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2706912488 0 0) 0x7f92d4001110 con 0x7f92e0155680 2023-10-06T04:32:10.723 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92deffd700 1 -- 172.21.15.123:0/2616600941 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92e00c2b80 con 0x7f92e0155680 2023-10-06T04:32:10.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92e4c8f700 1 -- 172.21.15.123:0/2616600941 >> v1:172.21.15.177:6789/0 conn(0x7f92e0155680 legacy=0x7f92e006ede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92e4c8f700 1 -- 172.21.15.123:0/2616600941 >> v1:172.21.15.123:6789/0 conn(0x7f92e0155a50 legacy=0x7f92e00c1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92e4c8f700 1 -- 172.21.15.123:0/2616600941 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e006da30 con 0x7f92e00cab00 2023-10-06T04:32:10.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92e4c8f700 1 --2- 172.21.15.123:0/2616600941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e00cab00 0x7f92e01607a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009e90 tx=0x7f92d8009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=904bd76a0529e232 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92deffd700 1 -- 172.21.15.123:0/2616600941 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d800e070 con 0x7f92e00cab00 2023-10-06T04:32:10.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92deffd700 1 -- 172.21.15.123:0/2616600941 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d800bb80 con 0x7f92e00cab00 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.720+0000 7f92deffd700 1 -- 172.21.15.123:0/2616600941 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d8012430 con 0x7f92e00cab00 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.721+0000 7f92e66f2700 1 -- 172.21.15.123:0/2616600941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e00cab00 msgr2=0x7f92e01607a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.721+0000 7f92e66f2700 1 --2- 172.21.15.123:0/2616600941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e00cab00 0x7f92e01607a0 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f92d8009e90 tx=0x7f92d8009780 comp rx=0 tx=0).stop 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.721+0000 7f92e66f2700 1 -- 172.21.15.123:0/2616600941 shutdown_connections 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.721+0000 7f92e66f2700 1 --2- 172.21.15.123:0/2616600941 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e00cab00 0x7f92e01607a0 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.721+0000 7f92e66f2700 1 -- 172.21.15.123:0/2616600941 >> 172.21.15.123:0/2616600941 conn(0x7f92e00cbf20 msgr2=0x7f92e00cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.721+0000 7f92e66f2700 1 -- 172.21.15.123:0/2616600941 shutdown_connections 2023-10-06T04:32:10.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.721+0000 7f92e66f2700 1 -- 172.21.15.123:0/2616600941 wait complete. 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.722+0000 7f92e66f2700 1 Processor -- start 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.722+0000 7f92e66f2700 1 -- start start 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e66f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92e00cab00 0x7f92e014ef20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e66f2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e0155680 0x7f92e0151690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e66f2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e0155a50 0x7f92e0153a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e66f2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0166a80 con 0x7f92e00cab00 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e66f2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e0168230 con 0x7f92e0155a50 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e66f2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e014f820 con 0x7f92e0155680 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92e00cab00 0x7f92e014ef20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92e00cab00 0x7f92e014ef20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50422/0 (socket says 172.21.15.123:50422) 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 -- 172.21.15.123:0/1221749583 learned_addr learned my addr 172.21.15.123:0/1221749583 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92df7fe700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e0155680 0x7f92e0151690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e4c8f700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e0155a50 0x7f92e0153a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 -- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e0155680 msgr2=0x7f92e0151690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e0155680 0x7f92e0151690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 -- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e0155a50 msgr2=0x7f92e0153a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e0155a50 0x7f92e0153a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92dffff700 1 -- 172.21.15.123:0/1221749583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d8009a20 con 0x7f92e00cab00 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.723+0000 7f92e4c8f700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e0155a50 0x7f92e0153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.724+0000 7f92dffff700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92e00cab00 0x7f92e014ef20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c7c0 tx=0x7f92d4007c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.724+0000 7f92c7fff700 1 -- 172.21.15.123:0/1221749583 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4011590 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.724+0000 7f92c7fff700 1 -- 172.21.15.123:0/1221749583 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d4011710 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.724+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92e014fb00 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.724+0000 7f92c7fff700 1 -- 172.21.15.123:0/1221749583 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d4011ad0 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.724+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e01500c0 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.725+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e00cbc50 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.726+0000 7f92c7fff700 1 -- 172.21.15.123:0/1221749583 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92d4025430 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.726+0000 7f92c7fff700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92cc066f30 0x7f92cc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.726+0000 7f92c7fff700 1 -- 172.21.15.123:0/1221749583 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f92d4084ac0 con 0x7f92e00cab00 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.726+0000 7f92df7fe700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92cc066f30 0x7f92cc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:10.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.726+0000 7f92df7fe700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92cc066f30 0x7f92cc0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92e0155040 tx=0x7f92d0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:10.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.730+0000 7f92c7fff700 1 -- 172.21.15.123:0/1221749583 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92d4051c50 con 0x7f92e00cab00 2023-10-06T04:32:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.909+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f92e006f4f0 con 0x7f92e00cab00 2023-10-06T04:32:10.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.910+0000 7f92c7fff700 1 -- 172.21.15.123:0/1221749583 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v23) v1 ==== 104+0+11 (secure 0 0 0) 0x7f92e006f4f0 con 0x7f92e00cab00 2023-10-06T04:32:10.911 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92cc066f30 msgr2=0x7f92cc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92cc066f30 0x7f92cc0693e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f92e0155040 tx=0x7f92d0009040 comp rx=0 tx=0).stop 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92e00cab00 msgr2=0x7f92e014ef20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92e00cab00 0x7f92e014ef20 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f92d400c7c0 tx=0x7f92d4007c80 comp rx=0 tx=0).stop 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 shutdown_connections 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92cc066f30 0x7f92cc0693e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92e00cab00 0x7f92e014ef20 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92e0155a50 0x7f92e0153a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 --2- 172.21.15.123:0/1221749583 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92e0155680 0x7f92e0151690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 >> 172.21.15.123:0/1221749583 conn(0x7f92e00cbf20 msgr2=0x7f92e014e6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:10.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 shutdown_connections 2023-10-06T04:32:10.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:10.912+0000 7f92e66f2700 1 -- 172.21.15.123:0/1221749583 wait complete. 2023-10-06T04:32:10.925 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:32:11.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.044+0000 7fb139d42700 1 Processor -- start 2023-10-06T04:32:11.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.044+0000 7fb139d42700 1 -- start start 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb139d42700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb134157430 0x7fb1340ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb139d42700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb1340c9150 0x7fb134159690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb139d42700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1340c9d70 con 0x7fb1340caf00 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb139d42700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb134157800 con 0x7fb1340c9150 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb139d42700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340c9590 con 0x7fb134157430 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb1340c9150 0x7fb134159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb1340c9150 0x7fb134159690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50426/0 (socket says 172.21.15.123:50426) 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 -- 172.21.15.123:0/4013969164 learned_addr learned my addr 172.21.15.123:0/4013969164 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 -- 172.21.15.123:0/4013969164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb134157430 msgr2=0x7fb1340ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:11.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb1337fe700 1 --2- 172.21.15.123:0/4013969164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb134157430 0x7fb1340ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 --2- 172.21.15.123:0/4013969164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb134157430 0x7fb1340ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 -- 172.21.15.123:0/4013969164 >> v1:172.21.15.177:6789/0 conn(0x7fb1340caf00 legacy=0x7fb1340c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 -- 172.21.15.123:0/4013969164 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134159200 con 0x7fb1340c9150 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb133fff700 1 --2- 172.21.15.123:0/4013969164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134159690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009f50 tx=0x7fb12c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8147a9230a83674 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb1327fc700 1 -- 172.21.15.123:0/4013969164 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c00e070 con 0x7fb1340c9150 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb1327fc700 1 -- 172.21.15.123:0/4013969164 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb12c00ba30 con 0x7fb1340c9150 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.045+0000 7fb1327fc700 1 -- 172.21.15.123:0/4013969164 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb12c0092e0 con 0x7fb1340c9150 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 -- 172.21.15.123:0/4013969164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 msgr2=0x7fb134159690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 --2- 172.21.15.123:0/4013969164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134159690 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fb12c009f50 tx=0x7fb12c009840 comp rx=0 tx=0).stop 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 -- 172.21.15.123:0/4013969164 shutdown_connections 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 --2- 172.21.15.123:0/4013969164 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134159690 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 --2- 172.21.15.123:0/4013969164 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb134157430 0x7fb1340ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 -- 172.21.15.123:0/4013969164 >> 172.21.15.123:0/4013969164 conn(0x7fb134097450 msgr2=0x7fb134067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:11.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 -- 172.21.15.123:0/4013969164 shutdown_connections 2023-10-06T04:32:11.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.046+0000 7fb139d42700 1 -- 172.21.15.123:0/4013969164 wait complete. 2023-10-06T04:32:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.047+0000 7fb139d42700 1 Processor -- start 2023-10-06T04:32:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.047+0000 7fb139d42700 1 -- start start 2023-10-06T04:32:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb139d42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134069e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb139d42700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1340caf00 0x7fb1341520a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.052 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb139d42700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb134157430 0x7fb1341525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb139d42700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340b03a0 con 0x7fb1340c9150 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb139d42700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb134169aa0 con 0x7fb134157430 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb139d42700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb134152b20 con 0x7fb1340caf00 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134069e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134069e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50440/0 (socket says 172.21.15.123:50440) 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 -- 172.21.15.123:0/2434150438 learned_addr learned my addr 172.21.15.123:0/2434150438 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb133fff700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb134157430 0x7fb1341525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb132ffd700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1340caf00 0x7fb1341520a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 -- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1340caf00 msgr2=0x7fb1341520a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1340caf00 0x7fb1341520a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 -- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb134157430 msgr2=0x7fb1341525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.053 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb134157430 0x7fb1341525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb1337fe700 1 -- 172.21.15.123:0/2434150438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb12c009ae0 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.048+0000 7fb132ffd700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1340caf00 0x7fb1341520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.049+0000 7fb1337fe700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134069e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00c8b0 tx=0x7fb11c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.049+0000 7fb12b7fe700 1 -- 172.21.15.123:0/2434150438 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c011600 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.049+0000 7fb12b7fe700 1 -- 172.21.15.123:0/2434150438 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb11c011760 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.049+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb134169cb0 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.049+0000 7fb12b7fe700 1 -- 172.21.15.123:0/2434150438 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb11c011b00 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.049+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb13416a040 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.050+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13406d840 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.050+0000 7fb12b7fe700 1 -- 172.21.15.123:0/2434150438 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb11c018360 con 0x7fb1340c9150 2023-10-06T04:32:11.054 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.050+0000 7fb12b7fe700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb114066f30 0x7fb1140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.051+0000 7fb12b7fe700 1 -- 172.21.15.123:0/2434150438 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb11c084390 con 0x7fb1340c9150 2023-10-06T04:32:11.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.051+0000 7fb132ffd700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb114066f30 0x7fb1140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.055 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.051+0000 7fb132ffd700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb114066f30 0x7fb1140693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb1340c80c0 tx=0x7fb124009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.056 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.055+0000 7fb12b7fe700 1 -- 172.21.15.123:0/2434150438 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb11c054f30 con 0x7fb1340c9150 2023-10-06T04:32:11.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.233+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1340c97b0 con 0x7fb1340c9150 2023-10-06T04:32:11.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.233+0000 7fb12b7fe700 1 -- 172.21.15.123:0/2434150438 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v23) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb11c054ac0 con 0x7fb1340c9150 2023-10-06T04:32:11.235 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:32:11.237 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb114066f30 msgr2=0x7fb1140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.237 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb114066f30 0x7fb1140693e0 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fb1340c80c0 tx=0x7fb124009040 comp rx=0 tx=0).stop 2023-10-06T04:32:11.237 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 msgr2=0x7fb134069e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.237 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134069e70 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fb11c00c8b0 tx=0x7fb11c008040 comp rx=0 tx=0).stop 2023-10-06T04:32:11.237 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 shutdown_connections 2023-10-06T04:32:11.238 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb114066f30 0x7fb1140693e0 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.238 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb1340c9150 0x7fb134069e70 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.238 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb134157430 0x7fb1341525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.238 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 --2- 172.21.15.123:0/2434150438 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb1340caf00 0x7fb1341520a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.238 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.235+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 >> 172.21.15.123:0/2434150438 conn(0x7fb134097450 msgr2=0x7fb134154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:11.238 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.236+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 shutdown_connections 2023-10-06T04:32:11.238 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.236+0000 7fb139d42700 1 -- 172.21.15.123:0/2434150438 wait complete. 2023-10-06T04:32:11.249 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:11.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.366+0000 7f8f6ea7f700 1 Processor -- start 2023-10-06T04:32:11.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.367+0000 7f8f6ea7f700 1 -- start start 2023-10-06T04:32:11.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.367+0000 7f8f6ea7f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f6806bb60 0x7f8f6815d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.367+0000 7f8f6ea7f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8f6815d550 0x7f8f68069530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.367+0000 7f8f6ea7f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f680690a0 con 0x7f8f6813f600 2023-10-06T04:32:11.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.367+0000 7f8f6ea7f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f6815dd40 con 0x7f8f6806bb60 2023-10-06T04:32:11.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.367+0000 7f8f6ea7f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f68068770 con 0x7f8f6815d550 2023-10-06T04:32:11.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.367+0000 7f8f67fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f6806bb60 0x7f8f6815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f67fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8f6806bb60 0x7f8f6815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50456/0 (socket says 172.21.15.123:50456) 2023-10-06T04:32:11.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f67fff700 1 -- 172.21.15.123:0/652490910 learned_addr learned my addr 172.21.15.123:0/652490910 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:11.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f6d01c700 1 --2- 172.21.15.123:0/652490910 >> v2:172.21.15.177:3300/0 conn(0x7f8f6815d550 0x7f8f68069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f67fff700 1 -- 172.21.15.123:0/652490910 >> v2:172.21.15.177:3300/0 conn(0x7f8f6815d550 msgr2=0x7f8f68069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:11.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f67fff700 1 --2- 172.21.15.123:0/652490910 >> v2:172.21.15.177:3300/0 conn(0x7f8f6815d550 0x7f8f68069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f67fff700 1 -- 172.21.15.123:0/652490910 >> v1:172.21.15.177:6789/0 conn(0x7f8f6813f600 legacy=0x7f8f6806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f67fff700 1 -- 172.21.15.123:0/652490910 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f68068a50 con 0x7f8f6806bb60 2023-10-06T04:32:11.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.368+0000 7f8f67fff700 1 --2- 172.21.15.123:0/652490910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6806bb60 0x7f8f6815d010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8f6806fe20 tx=0x7f8f54009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb7a04b181bbbf5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.369+0000 7f8f677fe700 1 -- 172.21.15.123:0/652490910 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f54009e70 con 0x7f8f6806bb60 2023-10-06T04:32:11.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.369+0000 7f8f677fe700 1 -- 172.21.15.123:0/652490910 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f5400b860 con 0x7f8f6806bb60 2023-10-06T04:32:11.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.369+0000 7f8f677fe700 1 -- 172.21.15.123:0/652490910 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400bc00 con 0x7f8f6806bb60 2023-10-06T04:32:11.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.369+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/652490910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6806bb60 msgr2=0x7f8f6815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.369+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/652490910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6806bb60 0x7f8f6815d010 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f8f6806fe20 tx=0x7f8f54009250 comp rx=0 tx=0).stop 2023-10-06T04:32:11.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.369+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/652490910 shutdown_connections 2023-10-06T04:32:11.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.369+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/652490910 >> v2:172.21.15.177:3300/0 conn(0x7f8f6815d550 0x7f8f68069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.370+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/652490910 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6806bb60 0x7f8f6815d010 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.370+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/652490910 >> 172.21.15.123:0/652490910 conn(0x7f8f680ce0c0 msgr2=0x7f8f680ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:11.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.370+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/652490910 shutdown_connections 2023-10-06T04:32:11.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.370+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/652490910 wait complete. 2023-10-06T04:32:11.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.371+0000 7f8f6ea7f700 1 Processor -- start 2023-10-06T04:32:11.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.371+0000 7f8f6ea7f700 1 -- start start 2023-10-06T04:32:11.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6ea7f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f6806bb60 0x7f8f68158440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6ea7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6813f600 0x7f8f680c2780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6ea7f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6815d550 0x7f8f680b1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6ea7f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680b0e70 con 0x7f8f6813f600 2023-10-06T04:32:11.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6ea7f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680b1130 con 0x7f8f6815d550 2023-10-06T04:32:11.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6ea7f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f680b1410 con 0x7f8f6806bb60 2023-10-06T04:32:11.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6813f600 0x7f8f680c2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6813f600 0x7f8f680c2780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50472/0 (socket says 172.21.15.123:50472) 2023-10-06T04:32:11.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 -- 172.21.15.123:0/1602556441 learned_addr learned my addr 172.21.15.123:0/1602556441 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:11.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6d01c700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6815d550 0x7f8f680b1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f6c81b700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f6806bb60 0x7f8f68158440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 -- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f6806bb60 msgr2=0x7f8f68158440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f6806bb60 0x7f8f68158440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 -- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6815d550 msgr2=0x7f8f680b1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6815d550 0x7f8f680b1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 -- 172.21.15.123:0/1602556441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f540094f0 con 0x7f8f6813f600 2023-10-06T04:32:11.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.372+0000 7f8f67fff700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6813f600 0x7f8f680c2780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8f68071290 tx=0x7f8f54017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.376+0000 7f8f64ff9700 1 -- 172.21.15.123:0/1602556441 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f5400bbe0 con 0x7f8f6813f600 2023-10-06T04:32:11.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.376+0000 7f8f64ff9700 1 -- 172.21.15.123:0/1602556441 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54011420 con 0x7f8f6813f600 2023-10-06T04:32:11.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.376+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f680b16f0 con 0x7f8f6813f600 2023-10-06T04:32:11.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.376+0000 7f8f64ff9700 1 -- 172.21.15.123:0/1602556441 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540117e0 con 0x7f8f6813f600 2023-10-06T04:32:11.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.376+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f68158980 con 0x7f8f6813f600 2023-10-06T04:32:11.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.377+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f680b1c10 con 0x7f8f6813f600 2023-10-06T04:32:11.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.377+0000 7f8f64ff9700 1 -- 172.21.15.123:0/1602556441 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8f54011c10 con 0x7f8f6813f600 2023-10-06T04:32:11.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.378+0000 7f8f64ff9700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8f4c066f30 0x7f8f4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.378+0000 7f8f64ff9700 1 -- 172.21.15.123:0/1602556441 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8f54085e30 con 0x7f8f6813f600 2023-10-06T04:32:11.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.378+0000 7f8f6c81b700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8f4c066f30 0x7f8f4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.378+0000 7f8f6c81b700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8f4c066f30 0x7f8f4c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c003a80 tx=0x7f8f5c007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.382+0000 7f8f64ff9700 1 -- 172.21.15.123:0/1602556441 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8f680b1c10 con 0x7f8f6813f600 2023-10-06T04:32:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.606+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f680b1c10 con 0x7f8f6813f600 2023-10-06T04:32:11.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.607+0000 7f8f64ff9700 1 -- 172.21.15.123:0/1602556441 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f8f680b1c10 con 0x7f8f6813f600 2023-10-06T04:32:11.609 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:11.609 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:32:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8f4c066f30 msgr2=0x7f8f4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8f4c066f30 0x7f8f4c0693e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c003a80 tx=0x7f8f5c007ec0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6813f600 msgr2=0x7f8f680c2780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6813f600 0x7f8f680c2780 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f8f68071290 tx=0x7f8f54017000 comp rx=0 tx=0).stop 2023-10-06T04:32:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 shutdown_connections 2023-10-06T04:32:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8f4c066f30 0x7f8f4c0693e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8f6815d550 0x7f8f680b1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8f6806bb60 0x7f8f68158440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 --2- 172.21.15.123:0/1602556441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8f6813f600 0x7f8f680c2780 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 >> 172.21.15.123:0/1602556441 conn(0x7f8f680ce0c0 msgr2=0x7f8f680ca4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:11.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 shutdown_connections 2023-10-06T04:32:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.609+0000 7f8f6ea7f700 1 -- 172.21.15.123:0/1602556441 wait complete. 2023-10-06T04:32:11.615 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:32:11.622 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:11.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.738+0000 7f5067dc5700 1 Processor -- start 2023-10-06T04:32:11.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.738+0000 7f5067dc5700 1 -- start start 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5067dc5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5060156f60 0x7f506015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5067dc5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 0x7f5060157da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5067dc5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5060157910 con 0x7f506015b290 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5067dc5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506006d240 con 0x7f5060156f60 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5067dc5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506006d520 con 0x7f50600ade40 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5065360700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f506015b290 0x7f50600ad730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39292/0 (socket says 172.21.15.123:39292) 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5065360700 1 -- 172.21.15.123:0/129182886 learned_addr learned my addr 172.21.15.123:0/129182886 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:11.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5065b61700 1 --2- 172.21.15.123:0/129182886 >> v2:172.21.15.177:3300/0 conn(0x7f5060156f60 0x7f506015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5066362700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 0x7f5060157da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1129989065 0 0) 0x7f5060157910 con 0x7f506015b290 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.739+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f504c003570 con 0x7f506015b290 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3302969767 0 0) 0x7f504c003570 con 0x7f506015b290 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 msgr2=0x7f5060157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 0x7f5060157da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 >> v2:172.21.15.177:3300/0 conn(0x7f5060156f60 msgr2=0x7f506015ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 --2- 172.21.15.123:0/129182886 >> v2:172.21.15.177:3300/0 conn(0x7f5060156f60 0x7f506015ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506006d800 con 0x7f506015b290 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f505c001470 con 0x7f506015b290 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 >> v1:172.21.15.123:6789/0 conn(0x7f506015b290 legacy=0x7f50600ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c003a60 0x7f504c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f504c004810 0x7f504c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 0x7f504c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506006d240 con 0x7f504c004810 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506006d520 con 0x7f504c003a60 2023-10-06T04:32:11.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f504c00ba60 con 0x7f50600ade40 2023-10-06T04:32:11.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065360700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f504c004810 0x7f504c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065b61700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c003a60 0x7f504c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5066362700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 0x7f504c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065360700 1 -- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 msgr2=0x7f504c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:11.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065360700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600ade40 0x7f504c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065360700 1 -- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c003a60 msgr2=0x7f504c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:11.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065360700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c003a60 0x7f504c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065360700 1 -- 172.21.15.123:0/129182886 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f504c003570 con 0x7f504c004810 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5065360700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f504c004810 0x7f504c004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c390 tx=0x7f505c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cfcd28000ce2173 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00cc20 con 0x7f504c004810 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c00cd80 con 0x7f504c004810 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.740+0000 7f5064b5f700 1 -- 172.21.15.123:0/129182886 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c005b50 con 0x7f504c004810 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.741+0000 7f5067dc5700 1 -- 172.21.15.123:0/129182886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f504c004810 msgr2=0x7f504c004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.741+0000 7f5067dc5700 1 --2- 172.21.15.123:0/129182886 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f504c004810 0x7f504c004c80 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c390 tx=0x7f505c001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:11.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.741+0000 7f5066362700 1 -- 172.21.15.123:0/129182886 reap_dead start 2023-10-06T04:32:11.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.741+0000 7f5067dc5700 1 -- 172.21.15.123:0/129182886 shutdown_connections 2023-10-06T04:32:11.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.741+0000 7f5067dc5700 1 -- 172.21.15.123:0/129182886 >> 172.21.15.123:0/129182886 conn(0x7f50600ce0c0 msgr2=0x7f50600ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:11.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.741+0000 7f5067dc5700 1 -- 172.21.15.123:0/129182886 shutdown_connections 2023-10-06T04:32:11.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.741+0000 7f5067dc5700 1 -- 172.21.15.123:0/129182886 wait complete. 2023-10-06T04:32:11.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.742+0000 7f5067dc5700 1 Processor -- start 2023-10-06T04:32:11.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.743+0000 7f5067dc5700 1 -- start start 2023-10-06T04:32:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.743+0000 7f5067dc5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c004810 0x7f50600c33d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.743+0000 7f5067dc5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50600c3910 0x7f50600c6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.743+0000 7f5067dc5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600c7410 0x7f50600c78b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.743+0000 7f5067dc5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506016aa00 con 0x7f50600c3910 2023-10-06T04:32:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.743+0000 7f5067dc5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50600c5430 con 0x7f504c004810 2023-10-06T04:32:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.743+0000 7f5067dc5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50600b1aa0 con 0x7f50600c7410 2023-10-06T04:32:11.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50600c3910 0x7f50600c6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50600c3910 0x7f50600c6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50488/0 (socket says 172.21.15.123:50488) 2023-10-06T04:32:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 -- 172.21.15.123:0/170987739 learned_addr learned my addr 172.21.15.123:0/170987739 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065b61700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c004810 0x7f50600c33d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5066362700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600c7410 0x7f50600c78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 -- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600c7410 msgr2=0x7f50600c78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600c7410 0x7f50600c78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 -- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c004810 msgr2=0x7f50600c33d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c004810 0x7f50600c33d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 -- 172.21.15.123:0/170987739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50600c7df0 con 0x7f50600c3910 2023-10-06T04:32:11.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5066362700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600c7410 0x7f50600c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5065360700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50600c3910 0x7f50600c6ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f505c001760 tx=0x7f505c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5055ffb700 1 -- 172.21.15.123:0/170987739 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c00cc20 con 0x7f50600c3910 2023-10-06T04:32:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5055ffb700 1 -- 172.21.15.123:0/170987739 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c0072e0 con 0x7f50600c3910 2023-10-06T04:32:11.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50600c4320 con 0x7f50600c3910 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5055ffb700 1 -- 172.21.15.123:0/170987739 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c007680 con 0x7f50600c3910 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.744+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50600c4040 con 0x7f50600c3910 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.745+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506006e690 con 0x7f50600c3910 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.746+0000 7f5055ffb700 1 -- 172.21.15.123:0/170987739 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f505c019030 con 0x7f50600c3910 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.746+0000 7f5055ffb700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5044066f30 0x7f50440693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.746+0000 7f5055ffb700 1 -- 172.21.15.123:0/170987739 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f505c085e40 con 0x7f50600c3910 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.746+0000 7f5065b61700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5044066f30 0x7f50440693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.746+0000 7f5065b61700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5044066f30 0x7f50440693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50600c4770 tx=0x7f50500091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:11.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.750+0000 7f5055ffb700 1 -- 172.21.15.123:0/170987739 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f505c052f50 con 0x7f50600c3910 2023-10-06T04:32:11.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.930+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f50600b1be0 con 0x7f50600c3910 2023-10-06T04:32:11.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.931+0000 7f5055ffb700 1 -- 172.21.15.123:0/170987739 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f505c056570 con 0x7f50600c3910 2023-10-06T04:32:11.933 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:11.934 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5044066f30 msgr2=0x7f50440693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5044066f30 0x7f50440693e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f50600c4770 tx=0x7f50500091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50600c3910 msgr2=0x7f50600c6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50600c3910 0x7f50600c6ed0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f505c001760 tx=0x7f505c015000 comp rx=0 tx=0).stop 2023-10-06T04:32:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 shutdown_connections 2023-10-06T04:32:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5044066f30 0x7f50440693e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f50600c7410 0x7f50600c78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f504c004810 0x7f50600c33d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 --2- 172.21.15.123:0/170987739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f50600c3910 0x7f50600c6ed0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 >> 172.21.15.123:0/170987739 conn(0x7f50600ce0c0 msgr2=0x7f50601553e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 shutdown_connections 2023-10-06T04:32:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:11.933+0000 7f5067dc5700 1 -- 172.21.15.123:0/170987739 wait complete. 2023-10-06T04:32:11.949 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:32:11.949 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:12.065 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.064+0000 7f6bdf895700 1 Processor -- start 2023-10-06T04:32:12.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.064+0000 7f6bdf895700 1 -- start start 2023-10-06T04:32:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdf895700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f6bd81560c0 0x7f6bd80c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdf895700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6bd806e960 0x7f6bd80ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdf895700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bd80c8580 con 0x7f6bd806f080 2023-10-06T04:32:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdf895700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd80c8860 con 0x7f6bd806e960 2023-10-06T04:32:12.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdf895700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6bd80c8b40 con 0x7f6bd81560c0 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdde32700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6bd806e960 0x7f6bd80ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdde32700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6bd806e960 0x7f6bd80ada60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50498/0 (socket says 172.21.15.123:50498) 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdde32700 1 -- 172.21.15.123:0/193384036 learned_addr learned my addr 172.21.15.123:0/193384036 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdd631700 1 --2- 172.21.15.123:0/193384036 >> v2:172.21.15.177:3300/0 conn(0x7f6bd81560c0 0x7f6bd80c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdde32700 1 -- 172.21.15.123:0/193384036 >> v2:172.21.15.177:3300/0 conn(0x7f6bd81560c0 msgr2=0x7f6bd80c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdde32700 1 --2- 172.21.15.123:0/193384036 >> v2:172.21.15.177:3300/0 conn(0x7f6bd81560c0 0x7f6bd80c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdde32700 1 -- 172.21.15.123:0/193384036 >> v1:172.21.15.177:6789/0 conn(0x7f6bd806f080 legacy=0x7f6bd806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.065+0000 7f6bdde32700 1 -- 172.21.15.123:0/193384036 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd80c8e20 con 0x7f6bd806e960 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdd631700 1 --2- 172.21.15.123:0/193384036 >> v2:172.21.15.177:3300/0 conn(0x7f6bd81560c0 0x7f6bd80c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdde32700 1 --2- 172.21.15.123:0/193384036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd80ada60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bd806e150 tx=0x7f6bd000ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e026c264a0d53d7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bcffff700 1 -- 172.21.15.123:0/193384036 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd0010070 con 0x7f6bd806e960 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bcffff700 1 -- 172.21.15.123:0/193384036 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bd000de90 con 0x7f6bd806e960 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bcffff700 1 -- 172.21.15.123:0/193384036 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bd000b320 con 0x7f6bd806e960 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdf895700 1 -- 172.21.15.123:0/193384036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 msgr2=0x7f6bd80ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdf895700 1 --2- 172.21.15.123:0/193384036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd80ada60 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f6bd806e150 tx=0x7f6bd000ba50 comp rx=0 tx=0).stop 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdf895700 1 -- 172.21.15.123:0/193384036 shutdown_connections 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdf895700 1 --2- 172.21.15.123:0/193384036 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd80ada60 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdf895700 1 --2- 172.21.15.123:0/193384036 >> v2:172.21.15.177:3300/0 conn(0x7f6bd81560c0 0x7f6bd80c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdf895700 1 -- 172.21.15.123:0/193384036 >> 172.21.15.123:0/193384036 conn(0x7f6bd80cda80 msgr2=0x7f6bd80c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.066+0000 7f6bdf895700 1 -- 172.21.15.123:0/193384036 shutdown_connections 2023-10-06T04:32:12.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.067+0000 7f6bdf895700 1 -- 172.21.15.123:0/193384036 wait complete. 2023-10-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 Processor -- start 2023-10-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 -- start start 2023-10-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd8169e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bd806f080 0x7f6bd80c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bd81560c0 0x7f6bd80aa8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd816a370 con 0x7f6bd806e960 2023-10-06T04:32:12.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd80c5110 con 0x7f6bd806f080 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.068+0000 7f6bdf895700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bd80c4500 con 0x7f6bd81560c0 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd8169e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd8169e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50504/0 (socket says 172.21.15.123:50504) 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 -- 172.21.15.123:0/634632909 learned_addr learned my addr 172.21.15.123:0/634632909 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdce30700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bd806f080 0x7f6bd80c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdde32700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bd81560c0 0x7f6bd80aa8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 -- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bd81560c0 msgr2=0x7f6bd80aa8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bd81560c0 0x7f6bd80aa8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 -- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bd806f080 msgr2=0x7f6bd80c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bd806f080 0x7f6bd80c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 -- 172.21.15.123:0/634632909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bd000bcf0 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdd631700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd8169e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800d860 tx=0x7f6bc8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bcd7fa700 1 -- 172.21.15.123:0/634632909 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc80125f0 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bcd7fa700 1 -- 172.21.15.123:0/634632909 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bc8012750 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bd80c47e0 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bcd7fa700 1 -- 172.21.15.123:0/634632909 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc8012ad0 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.069+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bd80c4da0 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.070+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bd80c53f0 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.071+0000 7f6bcd7fa700 1 -- 172.21.15.123:0/634632909 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6bc8014430 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.071+0000 7f6bcd7fa700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6bbc066f30 0x7f6bbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.071+0000 7f6bcd7fa700 1 -- 172.21.15.123:0/634632909 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6bc8084b20 con 0x7f6bd806e960 2023-10-06T04:32:12.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.071+0000 7f6bdce30700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6bbc066f30 0x7f6bbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.071+0000 7f6bdce30700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6bbc066f30 0x7f6bbc0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80ac4e0 tx=0x7f6bd4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.076 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.075+0000 7f6bcd7fa700 1 -- 172.21.15.123:0/634632909 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6bc8051c30 con 0x7f6bd806e960 2023-10-06T04:32:12.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.301+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6bd80ccb50 con 0x7f6bd806e960 2023-10-06T04:32:12.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.302+0000 7f6bcd7fa700 1 -- 172.21.15.123:0/634632909 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f6bc8055250 con 0x7f6bd806e960 2023-10-06T04:32:12.303 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:12.303 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:32:12.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6bbc066f30 msgr2=0x7f6bbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6bbc066f30 0x7f6bbc0693e0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f6bd80ac4e0 tx=0x7f6bd4009040 comp rx=0 tx=0).stop 2023-10-06T04:32:12.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 msgr2=0x7f6bd8169e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd8169e30 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f6bc800d860 tx=0x7f6bc8008040 comp rx=0 tx=0).stop 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 shutdown_connections 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6bbc066f30 0x7f6bbc0693e0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6bd806e960 0x7f6bd8169e30 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6bd81560c0 0x7f6bd80aa8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 --2- 172.21.15.123:0/634632909 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6bd806f080 0x7f6bd80c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 >> 172.21.15.123:0/634632909 conn(0x7f6bd80cda80 msgr2=0x7f6bd806b200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 shutdown_connections 2023-10-06T04:32:12.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.304+0000 7f6bdf895700 1 -- 172.21.15.123:0/634632909 wait complete. 2023-10-06T04:32:12.307 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:32:12.317 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:12.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.434+0000 7f685f853700 1 Processor -- start 2023-10-06T04:32:12.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.435+0000 7f685f853700 1 -- start start 2023-10-06T04:32:12.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.435+0000 7f685f853700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f685806bb60 0x7f685815d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.435+0000 7f685f853700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f685815d550 0x7f6858069530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.435+0000 7f685f853700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68580690a0 con 0x7f685813f600 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.435+0000 7f685f853700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f685815dd40 con 0x7f685806bb60 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.435+0000 7f685f853700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6858068770 con 0x7f685815d550 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685cdee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f685806bb60 0x7f685815d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685d5ef700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f685813f600 0x7f685806b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39298/0 (socket says 172.21.15.123:39298) 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685cdee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f685806bb60 0x7f685815d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50506/0 (socket says 172.21.15.123:50506) 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685d5ef700 1 -- 172.21.15.123:0/874446587 learned_addr learned my addr 172.21.15.123:0/874446587 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685ddf0700 1 --2- 172.21.15.123:0/874446587 >> v2:172.21.15.177:3300/0 conn(0x7f685815d550 0x7f6858069530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685cdee700 1 -- 172.21.15.123:0/874446587 >> v2:172.21.15.177:3300/0 conn(0x7f685815d550 msgr2=0x7f6858069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685cdee700 1 --2- 172.21.15.123:0/874446587 >> v2:172.21.15.177:3300/0 conn(0x7f685815d550 0x7f6858069530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685cdee700 1 -- 172.21.15.123:0/874446587 >> v1:172.21.15.123:6789/0 conn(0x7f685813f600 legacy=0x7f685806b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685cdee700 1 -- 172.21.15.123:0/874446587 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6858068a50 con 0x7f685806bb60 2023-10-06T04:32:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f684ffff700 1 -- 172.21.15.123:0/874446587 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4222768456 0 0) 0x7f68580690a0 con 0x7f685813f600 2023-10-06T04:32:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f685cdee700 1 --2- 172.21.15.123:0/874446587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685806bb60 0x7f685815d010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f685806fe20 tx=0x7f68540091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa5f69a412a6315b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f684ffff700 1 -- 172.21.15.123:0/874446587 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854011630 con 0x7f685806bb60 2023-10-06T04:32:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f684ffff700 1 -- 172.21.15.123:0/874446587 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6854011790 con 0x7f685806bb60 2023-10-06T04:32:12.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.436+0000 7f684ffff700 1 -- 172.21.15.123:0/874446587 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6854011b30 con 0x7f685806bb60 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 -- 172.21.15.123:0/874446587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685806bb60 msgr2=0x7f685815d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 --2- 172.21.15.123:0/874446587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685806bb60 0x7f685815d010 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f685806fe20 tx=0x7f68540091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 -- 172.21.15.123:0/874446587 shutdown_connections 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 --2- 172.21.15.123:0/874446587 >> v2:172.21.15.177:3300/0 conn(0x7f685815d550 0x7f6858069530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 --2- 172.21.15.123:0/874446587 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685806bb60 0x7f685815d010 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 -- 172.21.15.123:0/874446587 >> 172.21.15.123:0/874446587 conn(0x7f68580ce0c0 msgr2=0x7f68580ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 -- 172.21.15.123:0/874446587 shutdown_connections 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.437+0000 7f685f853700 1 -- 172.21.15.123:0/874446587 wait complete. 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.438+0000 7f685f853700 1 Processor -- start 2023-10-06T04:32:12.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.438+0000 7f685f853700 1 -- start start 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685f853700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f685806bb60 0x7f68580c35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685f853700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f685813f600 0x7f685813d9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685f853700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685815d550 0x7f685815c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685f853700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580c3c40 con 0x7f685815d550 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685f853700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68580c3f20 con 0x7f685813f600 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685f853700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68580c6140 con 0x7f685806bb60 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685815d550 0x7f685815c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685815d550 0x7f685815c1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50508/0 (socket says 172.21.15.123:50508) 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 -- 172.21.15.123:0/2346808174 learned_addr learned my addr 172.21.15.123:0/2346808174 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685cdee700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f685813f600 0x7f685813d9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685d5ef700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f685806bb60 0x7f68580c35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 -- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f685806bb60 msgr2=0x7f68580c35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f685806bb60 0x7f68580c35b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 -- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f685813f600 msgr2=0x7f685813d9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f685813f600 0x7f685813d9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 -- 172.21.15.123:0/2346808174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6854009420 con 0x7f685815d550 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.439+0000 7f685ddf0700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685815d550 0x7f685815c1b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6850009e90 tx=0x7f685000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.440+0000 7f684d7fa700 1 -- 172.21.15.123:0/2346808174 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f685000cb70 con 0x7f685815d550 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.440+0000 7f684d7fa700 1 -- 172.21.15.123:0/2346808174 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f685000f040 con 0x7f685815d550 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.440+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68580c6420 con 0x7f685815d550 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.440+0000 7f684d7fa700 1 -- 172.21.15.123:0/2346808174 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6850014480 con 0x7f685815d550 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.440+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f685815c6f0 con 0x7f685815d550 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.440+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f685806e690 con 0x7f685815d550 2023-10-06T04:32:12.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.441+0000 7f684d7fa700 1 -- 172.21.15.123:0/2346808174 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6850014830 con 0x7f685815d550 2023-10-06T04:32:12.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.441+0000 7f684d7fa700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f683c066f30 0x7f683c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.441+0000 7f684d7fa700 1 -- 172.21.15.123:0/2346808174 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f6850085040 con 0x7f685815d550 2023-10-06T04:32:12.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.441+0000 7f685d5ef700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f683c066f30 0x7f683c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.442+0000 7f685d5ef700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f683c066f30 0x7f683c0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6848007fd0 tx=0x7f6848007d30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.446+0000 7f684d7fa700 1 -- 172.21.15.123:0/2346808174 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6850052150 con 0x7f685815d550 2023-10-06T04:32:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.667+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f685815cca0 con 0x7f685815d550 2023-10-06T04:32:12.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.668+0000 7f684d7fa700 1 -- 172.21.15.123:0/2346808174 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4600 (secure 0 0 0) 0x7f6850055770 con 0x7f685815d550 2023-10-06T04:32:12.669 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:12.669 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4244,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/882397806","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":882397806},{"type":"v1","addr":"172.21.15.177:6834","nonce":882397806}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4285,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/3009534964","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":3009534964},{"type":"v1","addr":"172.21.15.177:6835","nonce":3009534964}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4290,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6835/2478191878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2478191878},{"type":"v1","addr":"172.21.15.123:6835","nonce":2478191878}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:31:34.419212+0000","modified":"2023-10-06T04:31:38.254397+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4293},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4293":{"gid":4293,"name":"c","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.123:6837/2210806135","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6836","nonce":2210806135},{"type":"v1","addr":"172.21.15.123:6837","nonce":2210806135}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[4],"metadata_pool":3,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":1}]} 2023-10-06T04:32:12.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f683c066f30 msgr2=0x7f683c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f683c066f30 0x7f683c0693e0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f6848007fd0 tx=0x7f6848007d30 comp rx=0 tx=0).stop 2023-10-06T04:32:12.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685815d550 msgr2=0x7f685815c1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685815d550 0x7f685815c1b0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f6850009e90 tx=0x7f685000b040 comp rx=0 tx=0).stop 2023-10-06T04:32:12.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 shutdown_connections 2023-10-06T04:32:12.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f683c066f30 0x7f683c0693e0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f685815d550 0x7f685815c1b0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.674 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f685806bb60 0x7f68580c35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.674 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 --2- 172.21.15.123:0/2346808174 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f685813f600 0x7f685813d9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.674 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 >> 172.21.15.123:0/2346808174 conn(0x7f68580ce0c0 msgr2=0x7f685813e550 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:12.674 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 shutdown_connections 2023-10-06T04:32:12.675 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.670+0000 7f685f853700 1 -- 172.21.15.123:0/2346808174 wait complete. 2023-10-06T04:32:12.675 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 8 2023-10-06T04:32:12.684 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:12.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.799+0000 7f8acf4ad700 1 Processor -- start 2023-10-06T04:32:12.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.800+0000 7f8acf4ad700 1 -- start start 2023-10-06T04:32:12.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.800+0000 7f8acf4ad700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ac8158f70 0x7f8ac81593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.800+0000 7f8acf4ad700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac8069a30 con 0x7f8ac80688c0 2023-10-06T04:32:12.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.800+0000 7f8acf4ad700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ac806a890 con 0x7f8ac8068c90 2023-10-06T04:32:12.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acf4ad700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ac8159930 con 0x7f8ac8158f70 2023-10-06T04:32:12.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acd249700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8ac80688c0 0x7f8ac80c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39304/0 (socket says 172.21.15.123:39304) 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acda4a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8ac8158f70 0x7f8ac81593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acd249700 1 -- 172.21.15.123:0/2657395645 learned_addr learned my addr 172.21.15.123:0/2657395645 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acda4a700 1 -- 172.21.15.123:0/2657395645 >> v1:172.21.15.177:6789/0 conn(0x7f8ac8068c90 legacy=0x7f8ac806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acda4a700 1 -- 172.21.15.123:0/2657395645 >> v1:172.21.15.123:6789/0 conn(0x7f8ac80688c0 legacy=0x7f8ac80c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acda4a700 1 -- 172.21.15.123:0/2657395645 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac8069c80 con 0x7f8ac8158f70 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8acda4a700 1 --2- 172.21.15.123:0/2657395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac8158f70 0x7f8ac81593f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009e60 tx=0x7f8ac0009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=792111126f4d2b57 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8abffff700 1 -- 172.21.15.123:0/2657395645 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac000e070 con 0x7f8ac8158f70 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8abffff700 1 -- 172.21.15.123:0/2657395645 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ac000bbf0 con 0x7f8ac8158f70 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.801+0000 7f8abffff700 1 -- 172.21.15.123:0/2657395645 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ac0012430 con 0x7f8ac8158f70 2023-10-06T04:32:12.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.802+0000 7f8acf4ad700 1 -- 172.21.15.123:0/2657395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac8158f70 msgr2=0x7f8ac81593f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.802+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/2657395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac8158f70 0x7f8ac81593f0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f8ac0009e60 tx=0x7f8ac0009780 comp rx=0 tx=0).stop 2023-10-06T04:32:12.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.802+0000 7f8acf4ad700 1 -- 172.21.15.123:0/2657395645 shutdown_connections 2023-10-06T04:32:12.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.802+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/2657395645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac8158f70 0x7f8ac81593f0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.802+0000 7f8acf4ad700 1 -- 172.21.15.123:0/2657395645 >> 172.21.15.123:0/2657395645 conn(0x7f8ac80c9c90 msgr2=0x7f8ac806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:12.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.802+0000 7f8acf4ad700 1 -- 172.21.15.123:0/2657395645 shutdown_connections 2023-10-06T04:32:12.805 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.802+0000 7f8acf4ad700 1 -- 172.21.15.123:0/2657395645 wait complete. 2023-10-06T04:32:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.803+0000 7f8acf4ad700 1 Processor -- start 2023-10-06T04:32:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.803+0000 7f8acf4ad700 1 -- start start 2023-10-06T04:32:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acf4ad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac80688c0 0x7f8ac8156450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acf4ad700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8068c90 0x7f8ac8169180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acf4ad700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ac8158f70 0x7f8ac8159ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acf4ad700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac816a1e0 con 0x7f8ac80688c0 2023-10-06T04:32:12.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acf4ad700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac8156990 con 0x7f8ac8068c90 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acf4ad700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ac815c020 con 0x7f8ac8158f70 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac80688c0 0x7f8ac8156450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac80688c0 0x7f8ac8156450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50520/0 (socket says 172.21.15.123:50520) 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 -- 172.21.15.123:0/61616671 learned_addr learned my addr 172.21.15.123:0/61616671 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acca48700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8068c90 0x7f8ac8169180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acda4a700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ac8158f70 0x7f8ac8159ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 -- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ac8158f70 msgr2=0x7f8ac8159ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ac8158f70 0x7f8ac8159ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 -- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8068c90 msgr2=0x7f8ac8169180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8068c90 0x7f8ac8169180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 -- 172.21.15.123:0/61616671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ac0009a20 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acca48700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8068c90 0x7f8ac8169180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8acd249700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac80688c0 0x7f8ac8156450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c7c0 tx=0x7f8ab8007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.804+0000 7f8abd7fa700 1 -- 172.21.15.123:0/61616671 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8005b30 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.805+0000 7f8abd7fa700 1 -- 172.21.15.123:0/61616671 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab8005cb0 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.805+0000 7f8abd7fa700 1 -- 172.21.15.123:0/61616671 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab8004a80 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.805+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ac815ba10 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.805+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ac815bcb0 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.806+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ac806cc70 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.806+0000 7f8abd7fa700 1 -- 172.21.15.123:0/61616671 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ab80070f0 con 0x7f8ac80688c0 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.807+0000 7f8abd7fa700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8aac066f30 0x7f8aac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:12.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.807+0000 7f8acca48700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8aac066f30 0x7f8aac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.807+0000 7f8abd7fa700 1 -- 172.21.15.123:0/61616671 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f8ab8083b10 con 0x7f8ac80688c0 2023-10-06T04:32:12.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.807+0000 7f8acca48700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8aac066f30 0x7f8aac0693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ac81597a0 tx=0x7f8ac4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:12.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.811+0000 7f8abd7fa700 1 -- 172.21.15.123:0/61616671 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ab8050ca0 con 0x7f8ac80688c0 2023-10-06T04:32:12.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.992+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8ac8069f60 con 0x7f8ac80688c0 2023-10-06T04:32:12.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.993+0000 7f8abd7fa700 1 -- 172.21.15.123:0/61616671 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v23) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f8ab8018070 con 0x7f8ac80688c0 2023-10-06T04:32:12.995 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:12.995 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:31:36.034933+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":3,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:31:32.045143+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"22","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":4,"pool_name":"cephfs_data","create_time":"2023-10-06T04:31:33.390421+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"23","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8aac066f30 msgr2=0x7f8aac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8aac066f30 0x7f8aac0693e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8ac81597a0 tx=0x7f8ac4009040 comp rx=0 tx=0).stop 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac80688c0 msgr2=0x7f8ac8156450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac80688c0 0x7f8ac8156450 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f8ab800c7c0 tx=0x7f8ab8007f70 comp rx=0 tx=0).stop 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 shutdown_connections 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8aac066f30 0x7f8aac0693e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ac80688c0 0x7f8ac8156450 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ac8158f70 0x7f8ac8159ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.995+0000 7f8acf4ad700 1 --2- 172.21.15.123:0/61616671 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ac8068c90 0x7f8ac8169180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:12.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.996+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 >> 172.21.15.123:0/61616671 conn(0x7f8ac80c9c90 msgr2=0x7f8ac80b16b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:12.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.996+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 shutdown_connections 2023-10-06T04:32:12.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:12.996+0000 7f8acf4ad700 1 -- 172.21.15.123:0/61616671 wait complete. 2023-10-06T04:32:13.009 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:32:13.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.126+0000 7fc297064700 1 Processor -- start 2023-10-06T04:32:13.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.126+0000 7fc297064700 1 -- start start 2023-10-06T04:32:13.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc297064700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc29006cd80 0x7fc29006bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc297064700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29015afd0 con 0x7fc2900ad640 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc297064700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc29006d360 con 0x7fc2901571b0 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc297064700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc29006dc00 con 0x7fc29006cd80 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28ffff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc29006cd80 0x7fc29006bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc294e00700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7fc2901571b0 0x7fc29015b460 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:45312/0 (socket says 172.21.15.123:45312) 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc295601700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc2900ad640 0x7fc2900ada50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39312/0 (socket says 172.21.15.123:39312) 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc294e00700 1 -- 172.21.15.123:0/3445949525 learned_addr learned my addr 172.21.15.123:0/3445949525 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747437507 0 0) 0x7fc29015afd0 con 0x7fc2900ad640 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc280003570 con 0x7fc2900ad640 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3203124951 0 0) 0x7fc280003570 con 0x7fc2900ad640 2023-10-06T04:32:13.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 >> v2:172.21.15.177:3300/0 conn(0x7fc29006cd80 msgr2=0x7fc29006bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 --2- 172.21.15.123:0/3445949525 >> v2:172.21.15.177:3300/0 conn(0x7fc29006cd80 0x7fc29006bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 >> v1:172.21.15.177:6789/0 conn(0x7fc2901571b0 legacy=0x7fc29015b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29006d640 con 0x7fc2900ad640 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fc2880017f0 con 0x7fc2900ad640 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 >> v1:172.21.15.123:6789/0 conn(0x7fc2900ad640 legacy=0x7fc2900ada50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc280003a60 0x7fc280003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc280004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.127+0000 7fc28f7fe700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2800051c0 0x7fc28000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29006dc00 con 0x7fc280004810 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc28000c0d0 con 0x7fc2800051c0 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc28000c290 con 0x7fc280003a60 2023-10-06T04:32:13.132 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28ffff700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc280004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc294e00700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc280003a60 0x7fc280003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc295601700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2800051c0 0x7fc28000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28ffff700 1 -- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc280003a60 msgr2=0x7fc280003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28ffff700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc280003a60 0x7fc280003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28ffff700 1 -- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2800051c0 msgr2=0x7fc28000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28ffff700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2800051c0 0x7fc28000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28ffff700 1 -- 172.21.15.123:0/3445949525 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc280003570 con 0x7fc280004810 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc295601700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc2800051c0 0x7fc28000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc295601700 1 -- 172.21.15.123:0/3445949525 reap_dead start 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28ffff700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc280004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009c00 tx=0x7fc27c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1818ddddb0b9168b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00e710 con 0x7fc280004810 2023-10-06T04:32:13.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc27c00e870 con 0x7fc280004810 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.128+0000 7fc28f7fe700 1 -- 172.21.15.123:0/3445949525 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc27c00ec30 con 0x7fc280004810 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.129+0000 7fc297064700 1 -- 172.21.15.123:0/3445949525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 msgr2=0x7fc280004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.129+0000 7fc297064700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc280004c80 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc27c009c00 tx=0x7fc27c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.129+0000 7fc297064700 1 -- 172.21.15.123:0/3445949525 shutdown_connections 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.129+0000 7fc297064700 1 --2- 172.21.15.123:0/3445949525 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc280004c80 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.129+0000 7fc297064700 1 -- 172.21.15.123:0/3445949525 >> 172.21.15.123:0/3445949525 conn(0x7fc290068130 msgr2=0x7fc2900ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.129+0000 7fc297064700 1 -- 172.21.15.123:0/3445949525 shutdown_connections 2023-10-06T04:32:13.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.129+0000 7fc297064700 1 -- 172.21.15.123:0/3445949525 wait complete. 2023-10-06T04:32:13.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.130+0000 7fc297064700 1 Processor -- start 2023-10-06T04:32:13.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.131+0000 7fc297064700 1 -- start start 2023-10-06T04:32:13.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.131+0000 7fc297064700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc29015c130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.131+0000 7fc297064700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29015df10 0x7fc290155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.131+0000 7fc297064700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29015e380 0x7fc2901563a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.131+0000 7fc297064700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc29006c070 con 0x7fc280004810 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.131+0000 7fc297064700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2901568e0 con 0x7fc29015e380 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.131+0000 7fc297064700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2900c47d0 con 0x7fc29015df10 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc29015c130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc29015c130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50532/0 (socket says 172.21.15.123:50532) 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 -- 172.21.15.123:0/1800945815 learned_addr learned my addr 172.21.15.123:0/1800945815 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc28ffff700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29015df10 0x7fc290155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc295601700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29015e380 0x7fc2901563a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 -- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29015df10 msgr2=0x7fc290155e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29015df10 0x7fc290155e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 -- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29015e380 msgr2=0x7fc2901563a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29015e380 0x7fc2901563a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 -- 172.21.15.123:0/1800945815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc29015b460 con 0x7fc280004810 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc28ffff700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29015df10 0x7fc290155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc294e00700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc29015c130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc284001730 tx=0x7fc284007de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc28cff9700 1 -- 172.21.15.123:0/1800945815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284011590 con 0x7fc280004810 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc28cff9700 1 -- 172.21.15.123:0/1800945815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc284011710 con 0x7fc280004810 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2900c4a20 con 0x7fc280004810 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc28cff9700 1 -- 172.21.15.123:0/1800945815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc284011ad0 con 0x7fc280004810 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.132+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc29015b730 con 0x7fc280004810 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.133+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc29006e6b0 con 0x7fc280004810 2023-10-06T04:32:13.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.133+0000 7fc28cff9700 1 -- 172.21.15.123:0/1800945815 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc28401d460 con 0x7fc280004810 2023-10-06T04:32:13.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.134+0000 7fc28cff9700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc274066f30 0x7fc2740693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.134+0000 7fc28cff9700 1 -- 172.21.15.123:0/1800945815 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fc2840846f0 con 0x7fc280004810 2023-10-06T04:32:13.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.134+0000 7fc28ffff700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc274066f30 0x7fc2740693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.134+0000 7fc28ffff700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc274066f30 0x7fc2740693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0091e0 tx=0x7fc27c01a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:13.140 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.138+0000 7fc28cff9700 1 -- 172.21.15.123:0/1800945815 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc2840518b0 con 0x7fc280004810 2023-10-06T04:32:13.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.364+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fc29004bb80 con 0x7fc280004810 2023-10-06T04:32:13.365 INFO:tasks.ceph.mon.a.smithi123.stderr:2023-10-06T04:32:13.364+0000 7fea90960700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:32:13.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.367+0000 7fc28cff9700 1 -- 172.21.15.123:0/1800945815 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7fc284054ed0 con 0x7fc280004810 2023-10-06T04:32:13.370 INFO:tasks.ceph.mds.c.smithi123.stderr: -2209> 2023-10-06T04:31:39.130+0000 7f98c7533700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:32:13.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc274066f30 msgr2=0x7fc2740693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc274066f30 0x7fc2740693e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fc27c0091e0 tx=0x7fc27c01a000 comp rx=0 tx=0).stop 2023-10-06T04:32:13.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 msgr2=0x7fc29015c130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc29015c130 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fc284001730 tx=0x7fc284007de0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 shutdown_connections 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc274066f30 0x7fc2740693e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc280004810 0x7fc29015c130 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc29015e380 0x7fc2901563a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 --2- 172.21.15.123:0/1800945815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc29015df10 0x7fc290155e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 >> 172.21.15.123:0/1800945815 conn(0x7fc290068130 msgr2=0x7fc2900af4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 shutdown_connections 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.370+0000 7fc297064700 1 -- 172.21.15.123:0/1800945815 wait complete. 2023-10-06T04:32:13.372 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:32:13.385 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:32:13.395 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2023-10-06T04:32:13.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.503+0000 7fb8f6d43700 1 Processor -- start 2023-10-06T04:32:13.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.503+0000 7fb8f6d43700 1 -- start start 2023-10-06T04:32:13.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8f6d43700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8340 0x7fb8f00a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8f6d43700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8f00c8710 0x7fb8f00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8f6d43700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb8f0068670 0x7fb8f0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8f6d43700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f0069050 con 0x7fb8f00c8710 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8f6d43700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8f0069aa0 con 0x7fb8f0068670 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8f6d43700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f006a340 con 0x7fb8f00c8340 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8effff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8f00c8710 0x7fb8f00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8effff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb8f00c8710 0x7fb8f00c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50554/0 (socket says 172.21.15.123:50554) 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.504+0000 7fb8effff700 1 -- 172.21.15.123:0/3209512124 learned_addr learned my addr 172.21.15.123:0/3209512124 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8f52e0700 1 --2- 172.21.15.123:0/3209512124 >> v2:172.21.15.177:3300/0 conn(0x7fb8f0068670 0x7fb8f0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8f4adf700 1 --2- 172.21.15.123:0/3209512124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8340 0x7fb8f00a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8effff700 1 -- 172.21.15.123:0/3209512124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8340 msgr2=0x7fb8f00a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8effff700 1 --2- 172.21.15.123:0/3209512124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8340 0x7fb8f00a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8effff700 1 -- 172.21.15.123:0/3209512124 >> v2:172.21.15.177:3300/0 conn(0x7fb8f0068670 msgr2=0x7fb8f0068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8effff700 1 --2- 172.21.15.123:0/3209512124 >> v2:172.21.15.177:3300/0 conn(0x7fb8f0068670 0x7fb8f0068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8effff700 1 -- 172.21.15.123:0/3209512124 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f00aa6f0 con 0x7fb8f00c8710 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8effff700 1 --2- 172.21.15.123:0/3209512124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8710 0x7fb8f00c8b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0099b0 tx=0x7fb8dc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59aa942735206c7e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8ef7fe700 1 -- 172.21.15.123:0/3209512124 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00b820 con 0x7fb8f00c8710 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8ef7fe700 1 -- 172.21.15.123:0/3209512124 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc00b980 con 0x7fb8f00c8710 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8ef7fe700 1 -- 172.21.15.123:0/3209512124 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc00bd20 con 0x7fb8f00c8710 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3209512124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8710 msgr2=0x7fb8f00c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.505+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3209512124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8710 0x7fb8f00c8b60 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fb8dc0099b0 tx=0x7fb8dc009250 comp rx=0 tx=0).stop 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.506+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3209512124 shutdown_connections 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.506+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3209512124 >> v2:172.21.15.177:3300/0 conn(0x7fb8f0068670 0x7fb8f0068b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.506+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3209512124 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8340 0x7fb8f00a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.506+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3209512124 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8710 0x7fb8f00c8b60 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.506+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3209512124 >> 172.21.15.123:0/3209512124 conn(0x7fb8f00c9ca0 msgr2=0x7fb8f006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:13.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.506+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3209512124 shutdown_connections 2023-10-06T04:32:13.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.506+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3209512124 wait complete. 2023-10-06T04:32:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.507+0000 7fb8f6d43700 1 Processor -- start 2023-10-06T04:32:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.507+0000 7fb8f6d43700 1 -- start start 2023-10-06T04:32:13.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.507+0000 7fb8f6d43700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f0068670 0x7fb8f015a550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f6d43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8340 0x7fb8f0158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f6d43700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8710 0x7fb8f0158740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f6d43700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00c2d90 con 0x7fb8f00c8340 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f6d43700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00c0100 con 0x7fb8f0068670 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f6d43700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8f00c0f60 con 0x7fb8f00c8710 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8340 0x7fb8f0158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8340 0x7fb8f0158200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50564/0 (socket says 172.21.15.123:50564) 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 -- 172.21.15.123:0/3219170027 learned_addr learned my addr 172.21.15.123:0/3219170027 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:13.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f4adf700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f0068670 0x7fb8f015a550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 -- 172.21.15.123:0/3219170027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8710 msgr2=0x7fb8f0158740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8710 0x7fb8f0158740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 -- 172.21.15.123:0/3219170027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f0068670 msgr2=0x7fb8f015a550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f0068670 0x7fb8f015a550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 -- 172.21.15.123:0/3219170027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc0094f0 con 0x7fb8f00c8340 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8effff700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8340 0x7fb8f0158200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb8f006d190 tx=0x7fb8dc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8ecff9700 1 -- 172.21.15.123:0/3219170027 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc011e70 con 0x7fb8f00c8340 2023-10-06T04:32:13.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8ecff9700 1 -- 172.21.15.123:0/3219170027 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8dc024440 con 0x7fb8f00c8340 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8ecff9700 1 -- 172.21.15.123:0/3219170027 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8dc024800 con 0x7fb8f00c8340 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8f00c0350 con 0x7fb8f00c8340 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.508+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8f00c0910 con 0x7fb8f00c8340 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.509+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8f00c10a0 con 0x7fb8f00c8340 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.509+0000 7fb8ecff9700 1 -- 172.21.15.123:0/3219170027 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb8dc01a420 con 0x7fb8f00c8340 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.510+0000 7fb8ecff9700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb8d4066f30 0x7fb8d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.510+0000 7fb8ecff9700 1 -- 172.21.15.123:0/3219170027 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fb8dc086690 con 0x7fb8f00c8340 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.510+0000 7fb8f4adf700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb8d4066f30 0x7fb8d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:13.515 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.510+0000 7fb8f4adf700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb8d4066f30 0x7fb8d40693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4006fd0 tx=0x7fb8e4008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:13.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.514+0000 7fb8ecff9700 1 -- 172.21.15.123:0/3219170027 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb8dc053750 con 0x7fb8f00c8340 2023-10-06T04:32:13.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:13.738+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb8f0069cf0 con 0x7fb8f00c8340 2023-10-06T04:32:14.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.382+0000 7fb8ecff9700 1 -- 172.21.15.123:0/3219170027 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb8dc056d70 con 0x7fb8f00c8340 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb8d4066f30 msgr2=0x7fb8d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb8d4066f30 0x7fb8d40693e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7fb8e4006fd0 tx=0x7fb8e4008040 comp rx=0 tx=0).stop 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8340 msgr2=0x7fb8f0158200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8340 0x7fb8f0158200 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fb8f006d190 tx=0x7fb8dc017000 comp rx=0 tx=0).stop 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 shutdown_connections 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb8d4066f30 0x7fb8d40693e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb8f00c8710 0x7fb8f0158740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:14.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb8f0068670 0x7fb8f015a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 --2- 172.21.15.123:0/3219170027 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb8f00c8340 0x7fb8f0158200 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 >> 172.21.15.123:0/3219170027 conn(0x7fb8f00c9ca0 msgr2=0x7fb8f00cbe70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 shutdown_connections 2023-10-06T04:32:14.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.384+0000 7fb8f6d43700 1 -- 172.21.15.123:0/3219170027 wait complete. 2023-10-06T04:32:14.398 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:32:14.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.558+0000 7fbfa9658700 1 Processor -- start 2023-10-06T04:32:14.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.558+0000 7fbfa9658700 1 -- start start 2023-10-06T04:32:14.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa9658700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbfa40caac0 0x7fbfa40ca070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:14.562 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa9658700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa4069760 con 0x7fbfa40cae90 2023-10-06T04:32:14.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa9658700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfa4068b50 con 0x7fbfa40c9290 2023-10-06T04:32:14.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa9658700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfa4068e30 con 0x7fbfa40caac0 2023-10-06T04:32:14.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa2ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbfa40caac0 0x7fbfa40ca070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:14.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa37fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbfa40cae90 0x7fbfa4069bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39332/0 (socket says 172.21.15.123:39332) 2023-10-06T04:32:14.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa37fe700 1 -- 172.21.15.123:0/1953482302 learned_addr learned my addr 172.21.15.123:0/1953482302 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:14.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa2ffd700 1 -- 172.21.15.123:0/1953482302 >> v1:172.21.15.177:6789/0 conn(0x7fbfa40c9290 legacy=0x7fbfa40b0f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:14.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa2ffd700 1 -- 172.21.15.123:0/1953482302 >> v1:172.21.15.123:6789/0 conn(0x7fbfa40cae90 legacy=0x7fbfa4069bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa2ffd700 1 -- 172.21.15.123:0/1953482302 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfa4069110 con 0x7fbfa40caac0 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa2ffd700 1 --2- 172.21.15.123:0/1953482302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa40caac0 0x7fbfa40ca070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf980098d0 tx=0x7fbf98009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a67e7f9c5be06f10 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.559+0000 7fbfa1ffb700 1 -- 172.21.15.123:0/1953482302 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf98009e70 con 0x7fbfa40caac0 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa1ffb700 1 -- 172.21.15.123:0/1953482302 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf98005c10 con 0x7fbfa40caac0 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa1ffb700 1 -- 172.21.15.123:0/1953482302 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9800b610 con 0x7fbfa40caac0 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa9658700 1 -- 172.21.15.123:0/1953482302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa40caac0 msgr2=0x7fbfa40ca070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa9658700 1 --2- 172.21.15.123:0/1953482302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa40caac0 0x7fbfa40ca070 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fbf980098d0 tx=0x7fbf98009120 comp rx=0 tx=0).stop 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa9658700 1 -- 172.21.15.123:0/1953482302 shutdown_connections 2023-10-06T04:32:14.564 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa9658700 1 --2- 172.21.15.123:0/1953482302 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa40caac0 0x7fbfa40ca070 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:14.565 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa9658700 1 -- 172.21.15.123:0/1953482302 >> 172.21.15.123:0/1953482302 conn(0x7fbfa4097440 msgr2=0x7fbfa4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:14.565 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa9658700 1 -- 172.21.15.123:0/1953482302 shutdown_connections 2023-10-06T04:32:14.565 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.560+0000 7fbfa9658700 1 -- 172.21.15.123:0/1953482302 wait complete. 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.562+0000 7fbfa9658700 1 Processor -- start 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.562+0000 7fbfa9658700 1 -- start start 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa9658700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfa40c9290 0x7fbfa40b3850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa9658700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfa40cae90 0x7fbfa406c560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa9658700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa406caa0 0x7fbfa4169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa9658700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa40b3d90 con 0x7fbfa406caa0 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa9658700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa406cf20 con 0x7fbfa40cae90 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa9658700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfa416aa00 con 0x7fbfa40c9290 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa406caa0 0x7fbfa4169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:14.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa406caa0 0x7fbfa4169cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50586/0 (socket says 172.21.15.123:50586) 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 -- 172.21.15.123:0/2520075132 learned_addr learned my addr 172.21.15.123:0/2520075132 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa27fc700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfa40cae90 0x7fbfa406c560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa2ffd700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfa40c9290 0x7fbfa40b3850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 -- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfa40c9290 msgr2=0x7fbfa40b3850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfa40c9290 0x7fbfa40b3850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 -- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfa40cae90 msgr2=0x7fbfa406c560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfa40cae90 0x7fbfa406c560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 -- 172.21.15.123:0/2520075132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf980093c0 con 0x7fbfa406caa0 2023-10-06T04:32:14.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa27fc700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfa40cae90 0x7fbfa406c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbfa37fe700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa406caa0 0x7fbfa4169cd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00cc20 tx=0x7fbf9c0019c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.563+0000 7fbf8affd700 1 -- 172.21.15.123:0/2520075132 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00a940 con 0x7fbfa406caa0 2023-10-06T04:32:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.564+0000 7fbf8affd700 1 -- 172.21.15.123:0/2520075132 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf9c00aaa0 con 0x7fbfa406caa0 2023-10-06T04:32:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.564+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfa40c6240 con 0x7fbfa406caa0 2023-10-06T04:32:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.564+0000 7fbf8affd700 1 -- 172.21.15.123:0/2520075132 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf9c00ae20 con 0x7fbfa406caa0 2023-10-06T04:32:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.564+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfa40c6800 con 0x7fbfa406caa0 2023-10-06T04:32:14.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.564+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfa406da90 con 0x7fbfa406caa0 2023-10-06T04:32:14.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.565+0000 7fbf8affd700 1 -- 172.21.15.123:0/2520075132 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbf9c007300 con 0x7fbfa406caa0 2023-10-06T04:32:14.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.565+0000 7fbf8affd700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbf90066f30 0x7fbf900693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:14.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.565+0000 7fbf8affd700 1 -- 172.21.15.123:0/2520075132 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fbf9c084a10 con 0x7fbfa406caa0 2023-10-06T04:32:14.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.565+0000 7fbfa2ffd700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbf90066f30 0x7fbf900693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:14.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.566+0000 7fbfa2ffd700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbf90066f30 0x7fbf900693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfa406f2c0 tx=0x7fbf98015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:14.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.569+0000 7fbf8affd700 1 -- 172.21.15.123:0/2520075132 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbf9c051b50 con 0x7fbfa406caa0 2023-10-06T04:32:14.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:14.748+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbfa40693f0 con 0x7fbfa406caa0 2023-10-06T04:32:15.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.390+0000 7fbf8affd700 1 -- 172.21.15.123:0/2520075132 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7fbf9c055170 con 0x7fbfa406caa0 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbf90066f30 msgr2=0x7fbf900693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbf90066f30 0x7fbf900693e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7fbfa406f2c0 tx=0x7fbf98015000 comp rx=0 tx=0).stop 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa406caa0 msgr2=0x7fbfa4169cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa406caa0 0x7fbfa4169cd0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7fbf9c00cc20 tx=0x7fbf9c0019c0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 shutdown_connections 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbf90066f30 0x7fbf900693e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfa406caa0 0x7fbfa4169cd0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfa40c9290 0x7fbfa40b3850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 --2- 172.21.15.123:0/2520075132 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfa40cae90 0x7fbfa406c560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.393+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 >> 172.21.15.123:0/2520075132 conn(0x7fbfa4097440 msgr2=0x7fbfa40cbc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:15.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.394+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 shutdown_connections 2023-10-06T04:32:15.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.394+0000 7fbfa9658700 1 -- 172.21.15.123:0/2520075132 wait complete. 2023-10-06T04:32:15.396 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:32:15.411 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:32:15.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.531+0000 7f5fbc72a700 1 Processor -- start 2023-10-06T04:32:15.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.531+0000 7f5fbc72a700 1 -- start start 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbc72a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5fb4159610 0x7f5fb4068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbc72a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5fb406a150 0x7f5fb40c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbc72a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fb40685e0 con 0x7f5fb40cc290 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbc72a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb4068fb0 con 0x7f5fb406a150 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbc72a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fb40632c0 con 0x7f5fb4159610 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5fb406a150 0x7f5fb40c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5fb406a150 0x7f5fb40c6dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50594/0 (socket says 172.21.15.123:50594) 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 -- 172.21.15.123:0/2072135756 learned_addr learned my addr 172.21.15.123:0/2072135756 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:15.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 -- 172.21.15.123:0/2072135756 >> v2:172.21.15.177:3300/0 conn(0x7f5fb4159610 msgr2=0x7f5fb4068a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:15.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fba4c6700 1 --2- 172.21.15.123:0/2072135756 >> v2:172.21.15.177:3300/0 conn(0x7f5fb4159610 0x7f5fb4068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:15.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 --2- 172.21.15.123:0/2072135756 >> v2:172.21.15.177:3300/0 conn(0x7f5fb4159610 0x7f5fb4068a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 -- 172.21.15.123:0/2072135756 >> v1:172.21.15.177:6789/0 conn(0x7f5fb40cc290 legacy=0x7f5fb4069a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:15.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 -- 172.21.15.123:0/2072135756 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40679d0 con 0x7f5fb406a150 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.532+0000 7f5fbacc7700 1 --2- 172.21.15.123:0/2072135756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb406a150 0x7f5fb40c6dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5fb406d160 tx=0x7f5fac009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be40441c61711614 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fb94c4700 1 -- 172.21.15.123:0/2072135756 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac00e070 con 0x7f5fb406a150 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fb94c4700 1 -- 172.21.15.123:0/2072135756 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac00ba10 con 0x7f5fb406a150 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fb94c4700 1 -- 172.21.15.123:0/2072135756 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0092e0 con 0x7f5fb406a150 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 -- 172.21.15.123:0/2072135756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb406a150 msgr2=0x7f5fb40c6dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/2072135756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb406a150 0x7f5fb40c6dc0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7f5fb406d160 tx=0x7f5fac009840 comp rx=0 tx=0).stop 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 -- 172.21.15.123:0/2072135756 shutdown_connections 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/2072135756 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb406a150 0x7f5fb40c6dc0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.537 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/2072135756 >> v2:172.21.15.177:3300/0 conn(0x7f5fb4159610 0x7f5fb4068a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 -- 172.21.15.123:0/2072135756 >> 172.21.15.123:0/2072135756 conn(0x7f5fb40cd250 msgr2=0x7f5fb40c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:15.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 -- 172.21.15.123:0/2072135756 shutdown_connections 2023-10-06T04:32:15.538 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.533+0000 7f5fbc72a700 1 -- 172.21.15.123:0/2072135756 wait complete. 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.534+0000 7f5fbc72a700 1 Processor -- start 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.534+0000 7f5fbc72a700 1 -- start start 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbc72a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb406a150 0x7f5fb40b0df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbc72a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb40cc290 0x7f5fb40c5e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbc72a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4159610 0x7f5fb40c7fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbc72a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40b2350 con 0x7f5fb4159610 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbc72a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40b1740 con 0x7f5fb406a150 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbc72a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fb40b1a20 con 0x7f5fb40cc290 2023-10-06T04:32:15.541 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4159610 0x7f5fb40c7fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4159610 0x7f5fb40c7fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50598/0 (socket says 172.21.15.123:50598) 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 -- 172.21.15.123:0/499440736 learned_addr learned my addr 172.21.15.123:0/499440736 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fba4c6700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb406a150 0x7f5fb40b0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 -- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb40cc290 msgr2=0x7f5fb40c5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fb9cc5700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb40cc290 0x7f5fb40c5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb40cc290 0x7f5fb40c5e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 -- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb406a150 msgr2=0x7f5fb40b0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb406a150 0x7f5fb40b0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fbacc7700 1 -- 172.21.15.123:0/499440736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fac009ae0 con 0x7f5fb4159610 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.535+0000 7f5fba4c6700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb406a150 0x7f5fb40b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:15.542 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.536+0000 7f5fbacc7700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4159610 0x7f5fb40c7fb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5fac003000 tx=0x7f5fac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.536+0000 7f5faa7fc700 1 -- 172.21.15.123:0/499440736 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac0032f0 con 0x7f5fb4159610 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.536+0000 7f5faa7fc700 1 -- 172.21.15.123:0/499440736 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fac012e60 con 0x7f5fb4159610 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.536+0000 7f5faa7fc700 1 -- 172.21.15.123:0/499440736 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fac01d670 con 0x7f5fb4159610 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.536+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fb40b1d00 con 0x7f5fb4159610 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.536+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fb40b1330 con 0x7f5fb4159610 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.537+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fb40b2500 con 0x7f5fb4159610 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.537+0000 7f5faa7fc700 1 -- 172.21.15.123:0/499440736 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5fac01daa0 con 0x7f5fb4159610 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.538+0000 7f5faa7fc700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5f98066f30 0x7f5f980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.538+0000 7f5fba4c6700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5f98066f30 0x7f5f980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:15.543 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.538+0000 7f5faa7fc700 1 -- 172.21.15.123:0/499440736 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5764+0+0 (secure 0 0 0) 0x7f5fac085ea0 con 0x7f5fb4159610 2023-10-06T04:32:15.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.538+0000 7f5fba4c6700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5f98066f30 0x7f5f980693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0006fd0 tx=0x7f5fb0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:15.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.542+0000 7f5faa7fc700 1 -- 172.21.15.123:0/499440736 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5fb40b2500 con 0x7f5fb4159610 2023-10-06T04:32:15.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:15.724+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f5fb40b2490 con 0x7f5fb4159610 2023-10-06T04:32:16.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.396+0000 7f5faa7fc700 1 -- 172.21.15.123:0/499440736 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f5fac0530d0 con 0x7f5fb4159610 2023-10-06T04:32:16.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5f98066f30 msgr2=0x7f5f980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:16.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5f98066f30 0x7f5f980693e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0006fd0 tx=0x7f5fb0008040 comp rx=0 tx=0).stop 2023-10-06T04:32:16.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4159610 msgr2=0x7f5fb40c7fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:16.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4159610 0x7f5fb40c7fb0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f5fac003000 tx=0x7f5fac017000 comp rx=0 tx=0).stop 2023-10-06T04:32:16.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 shutdown_connections 2023-10-06T04:32:16.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5f98066f30 0x7f5f980693e0 secure :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5fb0006fd0 tx=0x7f5fb0008040 comp rx=0 tx=0).stop 2023-10-06T04:32:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5fb4159610 0x7f5fb40c7fb0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5fb406a150 0x7f5fb40b0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.402+0000 7f5fbc72a700 1 --2- 172.21.15.123:0/499440736 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5fb40cc290 0x7f5fb40c5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.403+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 >> 172.21.15.123:0/499440736 conn(0x7f5fb40cd250 msgr2=0x7f5fb40c2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.403+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 shutdown_connections 2023-10-06T04:32:16.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.403+0000 7f5fbc72a700 1 -- 172.21.15.123:0/499440736 wait complete. 2023-10-06T04:32:16.406 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2023-10-06T04:32:16.421 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:32:16.421 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:32:16.422 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:32:16.422 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:16.422 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:32:16.422 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:32:16.422 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:32:16.422 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:16.423 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:32:16.423 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:32:16.423 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:32:16.423 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:16.423 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:32:16.424 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:32:16.424 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:32:16.424 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:16.432 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:32:16.432 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:32:16.433 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:32:16.433 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:32:16.434 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:32:16.434 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:32:16.435 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:32:16.435 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:32:16.475 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:32:16.475 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:16.477 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:32:16.477 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:32:16.477 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:32:16.477 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:16.478 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:16.478 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:16.478 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-10-06T04:32:16.555 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2023-10-06T04:32:16.555 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-10-06T04:32:16.556 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2023-10-06T04:32:16.556 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-10-06T04:32:16.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.603+0000 7f5d388c1700 1 Processor -- start 2023-10-06T04:32:16.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.604+0000 7f5d388c1700 1 -- start start 2023-10-06T04:32:16.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.604+0000 7f5d388c1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d340ceed0 0x7f5d340cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.604+0000 7f5d388c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340cb5d0 0x7f5d340c5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.604+0000 7f5d388c1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d340cc980 con 0x7f5d340b28d0 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.604+0000 7f5d388c1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d34159e40 con 0x7f5d340ceed0 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.604+0000 7f5d388c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340b2ca0 con 0x7f5d340cb5d0 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d340ceed0 0x7f5d340cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f5d340ceed0 0x7f5d340cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50630/0 (socket says 172.21.15.123:50630) 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 -- 172.21.15.123:0/2668287414 learned_addr learned my addr 172.21.15.123:0/2668287414 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d32d9d700 1 --2- 172.21.15.123:0/2668287414 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340cb5d0 0x7f5d340c5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 -- 172.21.15.123:0/2668287414 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340cb5d0 msgr2=0x7f5d340c5330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d3159a700 1 -- 172.21.15.123:0/2668287414 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2361070201 0 0) 0x7f5d340cc980 con 0x7f5d340b28d0 2023-10-06T04:32:16.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 --2- 172.21.15.123:0/2668287414 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340cb5d0 0x7f5d340c5330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 -- 172.21.15.123:0/2668287414 >> v1:172.21.15.177:6789/0 conn(0x7f5d340b28d0 legacy=0x7f5d340ce7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 -- 172.21.15.123:0/2668287414 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d340cbd70 con 0x7f5d340ceed0 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d32d9d700 1 --2- 172.21.15.123:0/2668287414 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340cb5d0 0x7f5d340c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d31d9b700 1 --2- 172.21.15.123:0/2668287414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d340cb090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d240099b0 tx=0x7f5d24009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92b3b725deadc502 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d3159a700 1 -- 172.21.15.123:0/2668287414 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d24009ea0 con 0x7f5d340ceed0 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d3159a700 1 -- 172.21.15.123:0/2668287414 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2400b860 con 0x7f5d340ceed0 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.605+0000 7f5d3159a700 1 -- 172.21.15.123:0/2668287414 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2400bc00 con 0x7f5d340ceed0 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 -- 172.21.15.123:0/2668287414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 msgr2=0x7f5d340cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 --2- 172.21.15.123:0/2668287414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d340cb090 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f5d240099b0 tx=0x7f5d24009250 comp rx=0 tx=0).stop 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 -- 172.21.15.123:0/2668287414 shutdown_connections 2023-10-06T04:32:16.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 --2- 172.21.15.123:0/2668287414 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340cb5d0 0x7f5d340c5330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 --2- 172.21.15.123:0/2668287414 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d340cb090 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 -- 172.21.15.123:0/2668287414 >> 172.21.15.123:0/2668287414 conn(0x7f5d340995a0 msgr2=0x7f5d34063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:16.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 -- 172.21.15.123:0/2668287414 shutdown_connections 2023-10-06T04:32:16.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.606+0000 7f5d388c1700 1 -- 172.21.15.123:0/2668287414 wait complete. 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.607+0000 7f5d388c1700 1 Processor -- start 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.607+0000 7f5d388c1700 1 -- start start 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340b28d0 0x7f5d340c8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340cb5d0 0x7f5d340c9c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d34156e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340b30f0 con 0x7f5d340ceed0 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340b3c70 con 0x7f5d340cb5d0 2023-10-06T04:32:16.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d340b3340 con 0x7f5d340b28d0 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d34156e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d34156e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50646/0 (socket says 172.21.15.123:50646) 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 -- 172.21.15.123:0/4026928617 learned_addr learned my addr 172.21.15.123:0/4026928617 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d3259c700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340b28d0 0x7f5d340c8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 -- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340b28d0 msgr2=0x7f5d340c8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d31d9b700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340cb5d0 0x7f5d340c9c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340b28d0 0x7f5d340c8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 -- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340cb5d0 msgr2=0x7f5d340c9c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340cb5d0 0x7f5d340c9c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 -- 172.21.15.123:0/4026928617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d2c009a20 con 0x7f5d340ceed0 2023-10-06T04:32:16.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d3259c700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340b28d0 0x7f5d340c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d32d9d700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d34156e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009fc0 tx=0x7f5d2c00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d227fc700 1 -- 172.21.15.123:0/4026928617 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c009d20 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d227fc700 1 -- 172.21.15.123:0/4026928617 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d2c00f040 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d240094f0 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d227fc700 1 -- 172.21.15.123:0/4026928617 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d2c014480 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.608+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d340b3980 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.609+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d340ca240 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.610+0000 7f5d227fc700 1 -- 172.21.15.123:0/4026928617 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5d2c003020 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.610+0000 7f5d227fc700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d14066f30 0x7f5d140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.610+0000 7f5d227fc700 1 -- 172.21.15.123:0/4026928617 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(26..26 src has 1..26) v4 ==== 5292+0+0 (secure 0 0 0) 0x7f5d2c084da0 con 0x7f5d340ceed0 2023-10-06T04:32:16.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.610+0000 7f5d3259c700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d14066f30 0x7f5d140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:16.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.610+0000 7f5d3259c700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d14066f30 0x7f5d140693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d28001860 tx=0x7f5d28009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:16.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.614+0000 7f5d227fc700 1 -- 172.21.15.123:0/4026928617 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5d2c055c40 con 0x7f5d340ceed0 2023-10-06T04:32:16.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:16.809+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5d340654b0 con 0x7f5d340ceed0 2023-10-06T04:32:17.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.405+0000 7f5d227fc700 1 -- 172.21.15.123:0/4026928617 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v28) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5d2c0557d0 con 0x7f5d340ceed0 2023-10-06T04:32:17.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d14066f30 msgr2=0x7f5d140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d14066f30 0x7f5d140693e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f5d28001860 tx=0x7f5d28009040 comp rx=0 tx=0).stop 2023-10-06T04:32:17.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 msgr2=0x7f5d34156e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d34156e40 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f5d2c009fc0 tx=0x7f5d2c00b040 comp rx=0 tx=0).stop 2023-10-06T04:32:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 shutdown_connections 2023-10-06T04:32:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5d14066f30 0x7f5d140693e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5d340ceed0 0x7f5d34156e40 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5d340b28d0 0x7f5d340c8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 --2- 172.21.15.123:0/4026928617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5d340cb5d0 0x7f5d340c9c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.408+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 >> 172.21.15.123:0/4026928617 conn(0x7f5d340995a0 msgr2=0x7f5d34158da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:17.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.409+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 shutdown_connections 2023-10-06T04:32:17.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.409+0000 7f5d388c1700 1 -- 172.21.15.123:0/4026928617 wait complete. 2023-10-06T04:32:17.411 INFO:teuthology.orchestra.run.smithi123.stderr: removed all blocklist entries 2023-10-06T04:32:17.424 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-10-06T04:32:17.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.542+0000 7f900bf55700 1 Processor -- start 2023-10-06T04:32:17.544 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.542+0000 7f900bf55700 1 -- start start 2023-10-06T04:32:17.547 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.543+0000 7f900bf55700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9004158f70 0x7f90041593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.543+0000 7f900bf55700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9004069a30 con 0x7f9004068c90 2023-10-06T04:32:17.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.543+0000 7f900bf55700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f900406a890 con 0x7f90040688c0 2023-10-06T04:32:17.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.543+0000 7f900bf55700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9004159930 con 0x7f9004158f70 2023-10-06T04:32:17.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.543+0000 7f90094f0700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9004068c90 0x7f900406e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39348/0 (socket says 172.21.15.123:39348) 2023-10-06T04:32:17.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.543+0000 7f90094f0700 1 -- 172.21.15.123:0/3960394572 learned_addr learned my addr 172.21.15.123:0/3960394572 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:17.548 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 --2- 172.21.15.123:0/3960394572 >> v2:172.21.15.177:3300/0 conn(0x7f9004158f70 0x7f90041593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 767453013 0 0) 0x7f9004069a30 con 0x7f9004068c90 2023-10-06T04:32:17.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ff0003570 con 0x7f9004068c90 2023-10-06T04:32:17.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1939839913 0 0) 0x7f8ff0003570 con 0x7f9004068c90 2023-10-06T04:32:17.550 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 >> v2:172.21.15.177:3300/0 conn(0x7f9004158f70 msgr2=0x7f90041593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:17.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 --2- 172.21.15.123:0/3960394572 >> v2:172.21.15.177:3300/0 conn(0x7f9004158f70 0x7f90041593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 >> v1:172.21.15.177:6789/0 conn(0x7f90040688c0 legacy=0x7f90040c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9004069c80 con 0x7f9004068c90 2023-10-06T04:32:17.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f8ff8001470 con 0x7f9004068c90 2023-10-06T04:32:17.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 >> v1:172.21.15.123:6789/0 conn(0x7f9004068c90 legacy=0x7f900406e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff0003a60 0x7f8ff0003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.551 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff0004810 0x7f8ff0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff00051c0 0x7f8ff000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9004159930 con 0x7f8ff00051c0 2023-10-06T04:32:17.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff000c050 con 0x7f8ff0003a60 2023-10-06T04:32:17.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ff000c210 con 0x7f8ff0004810 2023-10-06T04:32:17.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff00051c0 0x7f8ff000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.552 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f9009cf1700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff0003a60 0x7f8ff0003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f90094f0700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff0004810 0x7f8ff0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 -- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff0004810 msgr2=0x7f8ff0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff0004810 0x7f8ff0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 -- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff0003a60 msgr2=0x7f8ff0003e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8ff0003a60 0x7f8ff0003e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 -- 172.21.15.123:0/3960394572 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ff0003570 con 0x7f8ff00051c0 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f90094f0700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff0004810 0x7f8ff0004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.544+0000 7f900a4f2700 1 -- 172.21.15.123:0/3960394572 reap_dead start 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900a4f2700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff00051c0 0x7f8ff000ba60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e210 tx=0x7f8ffc00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=882c2b74f703c60b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00eac0 con 0x7f8ff00051c0 2023-10-06T04:32:17.553 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc00f040 con 0x7f8ff00051c0 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f9008cef700 1 -- 172.21.15.123:0/3960394572 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00ee70 con 0x7f8ff00051c0 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900bf55700 1 -- 172.21.15.123:0/3960394572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff00051c0 msgr2=0x7f8ff000ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900bf55700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff00051c0 0x7f8ff000ba60 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00e210 tx=0x7f8ffc00b040 comp rx=0 tx=0).stop 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900bf55700 1 -- 172.21.15.123:0/3960394572 shutdown_connections 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900bf55700 1 --2- 172.21.15.123:0/3960394572 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8ff00051c0 0x7f8ff000ba60 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900bf55700 1 -- 172.21.15.123:0/3960394572 >> 172.21.15.123:0/3960394572 conn(0x7f90040c9c90 msgr2=0x7f900406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900bf55700 1 -- 172.21.15.123:0/3960394572 shutdown_connections 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.545+0000 7f900bf55700 1 -- 172.21.15.123:0/3960394572 wait complete. 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.546+0000 7f900bf55700 1 Processor -- start 2023-10-06T04:32:17.554 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.546+0000 7f900bf55700 1 -- start start 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900bf55700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff00051c0 0x7f90040c5320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900bf55700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90040b1dc0 0x7f9004169900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900bf55700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90040c3840 0x7f90040aff00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900bf55700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f90040af740 con 0x7f90040c3840 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900bf55700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f900416a270 con 0x7f90040b1dc0 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900bf55700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f900416a550 con 0x7f8ff00051c0 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900a4f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90040c3840 0x7f90040aff00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900a4f2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90040c3840 0x7f90040aff00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50656/0 (socket says 172.21.15.123:50656) 2023-10-06T04:32:17.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f900a4f2700 1 -- 172.21.15.123:0/2708923683 learned_addr learned my addr 172.21.15.123:0/2708923683 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:17.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f9009cf1700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff00051c0 0x7f90040c5320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.547+0000 7f90094f0700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90040b1dc0 0x7f9004169900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900a4f2700 1 -- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff00051c0 msgr2=0x7f90040c5320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900a4f2700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff00051c0 0x7f90040c5320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900a4f2700 1 -- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90040b1dc0 msgr2=0x7f9004169900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900a4f2700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90040b1dc0 0x7f9004169900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900a4f2700 1 -- 172.21.15.123:0/2708923683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f90040af9e0 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900a4f2700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90040c3840 0x7f90040aff00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f90040cc6d0 tx=0x7f8ffc019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f8ff5ffb700 1 -- 172.21.15.123:0/2708923683 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc01d650 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f8ff5ffb700 1 -- 172.21.15.123:0/2708923683 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ffc01d7d0 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f90040a9b60 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f8ff5ffb700 1 -- 172.21.15.123:0/2708923683 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc01dbd0 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.548+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f90040c5860 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.549+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f90040afbd0 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.549+0000 7f8ff5ffb700 1 -- 172.21.15.123:0/2708923683 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ffc003020 con 0x7f90040c3840 2023-10-06T04:32:17.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.550+0000 7f8ff5ffb700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8fe8066f30 0x7f8fe80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.550+0000 7f8ff5ffb700 1 -- 172.21.15.123:0/2708923683 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8ffc085db0 con 0x7f90040c3840 2023-10-06T04:32:17.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.550+0000 7f9009cf1700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8fe8066f30 0x7f8fe80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.550+0000 7f9009cf1700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8fe8066f30 0x7f8fe80693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9000000c00 tx=0x7f9000007d10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:17.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.554+0000 7f8ff5ffb700 1 -- 172.21.15.123:0/2708923683 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8ffc053210 con 0x7f90040c3840 2023-10-06T04:32:17.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.810+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9004069f60 con 0x7f90040c3840 2023-10-06T04:32:17.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.810+0000 7f8ff5ffb700 1 -- 172.21.15.123:0/2708923683 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v2) v1 ==== 80+0+6645 (secure 0 0 0) 0x7f8ffc05ae60 con 0x7f90040c3840 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "auth_dump": [ 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.a", 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBrjR9ln+DxMxAAUlHVQYS9oN2DiGRGYWQ3qg==", 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2023-10-06T04:32:17.812 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.b", 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBsjR9ljvRFBBAAk1yK46SI75Y+WxBrG+kPQQ==", 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2023-10-06T04:32:17.813 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.c", 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBrjR9l8lYJOxAA3cNOazP2iJLFQgSr92kLzA==", 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.814 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mds.d", 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBsjR9lRotZCxAAqXZ8VxmtgnlDE2xbHxwvlQ==", 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.815 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.0", 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBvjR9l0WieKRAAsku6SpM5TRo7hMMZNL24ZA==", 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.1", 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQByjR9lgYsMKBAAFEXHSp6wWEg/EDfA3OstaQ==", 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.816 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.2", 2023-10-06T04:32:17.817 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB1jR9l7T6WJhAA5PQEt56J46fTUUJGKsEvxw==", 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.3", 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB4jR9lNwGBJRAAZif6CdUCJ3upDIINQ4Y0Lw==", 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.818 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.819 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.4", 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQB+jR9lbiROJhAAJbUY2WVm0VtRp2OAatPqEA==", 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.5", 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCBjR9lpNCFJBAAlrytO/eUYhhp4ZgL6jVrtA==", 2023-10-06T04:32:17.820 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.6", 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCEjR9ld1GoIhAAh5yLDoZ5i3yGfZ6KGp/sug==", 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.821 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "osd.7", 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCHjR9lUpZoIRAAVD5W176MnYqzrAkZVYMVuQ==", 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow profile osd", 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile osd", 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.822 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.0", 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBsjR9laHCGDxAAVQanXVVoF5JlcwGfxMHDDQ==", 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.823 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.1", 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBsjR9lwuE7GBAA9Dxl3aHJegzlIbs9znobYQ==", 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.2", 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBsjR9l5/0JFRAAk6plb7Uw0yiJ2JRHEMSHqA==", 2023-10-06T04:32:17.824 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.3", 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBsjR9lLA3QHRAADHdP6xY/vy9W5bGELFNMoA==", 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.825 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow", 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow r", 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow rw", 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow rwx" 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.admin", 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBrjR9lWqNpFxAADiCbl3jCwreqW+2M0+0PAA==", 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.826 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "mgr": "allow *", 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow *", 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-mds", 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCNjR9llyOYGRAAQnINgtb+pvliDHIVH/bYOQ==", 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.827 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-mds" 2023-10-06T04:32:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-mgr", 2023-10-06T04:32:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCNjR9lpy2YGRAAWpkqt8McKp5UtIF6odD45A==", 2023-10-06T04:32:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-mgr" 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-osd", 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCNjR9lPDWYGRAAQnwlrzfKpKc4IwH4r3yLBQ==", 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-osd" 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rbd", 2023-10-06T04:32:17.829 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCNjR9loTyYGRAAtyb/nBXOr6vt32uKAWqviA==", 2023-10-06T04:32:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rbd" 2023-10-06T04:32:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.830 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCNjR9lDEeYGRAADHTcATrkSDx2B08SV4G9kg==", 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "client.bootstrap-rgw", 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQCNjR9lKU+YGRAA6g0jvSWWulpTphpcDECqYw==", 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile bootstrap-rgw" 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.831 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mgr.x", 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBrjR9lWKDSMBAAmtSOktWG9AchefQzbynbFw==", 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile mgr", 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "entity": "mgr.y", 2023-10-06T04:32:17.832 INFO:teuthology.orchestra.run.smithi123.stdout: "key": "AQBrjR9llNTxLBAAxlUBcZl4y4U9h573EW2L5g==", 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "caps": { 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "mds": "allow *", 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "mon": "allow profile mgr", 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: "osd": "allow *" 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout: ] 2023-10-06T04:32:17.833 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8fe8066f30 msgr2=0x7f8fe80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8fe8066f30 0x7f8fe80693e0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f9000000c00 tx=0x7f9000007d10 comp rx=0 tx=0).stop 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90040c3840 msgr2=0x7f90040aff00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90040c3840 0x7f90040aff00 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f90040cc6d0 tx=0x7f8ffc019000 comp rx=0 tx=0).stop 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 shutdown_connections 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8fe8066f30 0x7f8fe80693e0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f90040c3840 0x7f90040aff00 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8ff00051c0 0x7f90040c5320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 --2- 172.21.15.123:0/2708923683 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f90040b1dc0 0x7f9004169900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.835 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.812+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 >> 172.21.15.123:0/2708923683 conn(0x7f90040c9c90 msgr2=0x7f900406e430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:17.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.813+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 shutdown_connections 2023-10-06T04:32:17.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.813+0000 7f900bf55700 1 -- 172.21.15.123:0/2708923683 wait complete. 2023-10-06T04:32:17.836 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-10-06T04:32:17.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.953+0000 7f722fc4d700 1 Processor -- start 2023-10-06T04:32:17.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.953+0000 7f722fc4d700 1 -- start start 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722fc4d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72281594a0 0x7f72280c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722fc4d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f7228068d80 0x7f722806e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722fc4d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72280cad40 con 0x7f722806e450 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722fc4d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72280ca6a0 con 0x7f72281594a0 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722fc4d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72280cb550 con 0x7f7228068d80 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72281594a0 0x7f72280c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f72281594a0 0x7f72280c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50672/0 (socket says 172.21.15.123:50672) 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 -- 172.21.15.123:0/1575966422 learned_addr learned my addr 172.21.15.123:0/1575966422 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722e1ea700 1 --2- 172.21.15.123:0/1575966422 >> v2:172.21.15.177:3300/0 conn(0x7f7228068d80 0x7f722806e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 -- 172.21.15.123:0/1575966422 >> v2:172.21.15.177:3300/0 conn(0x7f7228068d80 msgr2=0x7f722806e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:17.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 --2- 172.21.15.123:0/1575966422 >> v2:172.21.15.177:3300/0 conn(0x7f7228068d80 0x7f722806e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 -- 172.21.15.123:0/1575966422 >> v1:172.21.15.177:6789/0 conn(0x7f722806e450 legacy=0x7f7228068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 -- 172.21.15.123:0/1575966422 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f722806a920 con 0x7f72281594a0 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722d9e9700 1 --2- 172.21.15.123:0/1575966422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72281594a0 0x7f72280c6dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f722806d9e0 tx=0x7f7218009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d567f203e0ec0a4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722c9e7700 1 -- 172.21.15.123:0/1575966422 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800b820 con 0x7f72281594a0 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722c9e7700 1 -- 172.21.15.123:0/1575966422 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721800b980 con 0x7f72281594a0 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.954+0000 7f722c9e7700 1 -- 172.21.15.123:0/1575966422 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721800bd20 con 0x7f72281594a0 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 -- 172.21.15.123:0/1575966422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72281594a0 msgr2=0x7f72280c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 --2- 172.21.15.123:0/1575966422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72281594a0 0x7f72280c6dd0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f722806d9e0 tx=0x7f7218009250 comp rx=0 tx=0).stop 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 -- 172.21.15.123:0/1575966422 shutdown_connections 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 --2- 172.21.15.123:0/1575966422 >> v2:172.21.15.177:3300/0 conn(0x7f7228068d80 0x7f722806e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 --2- 172.21.15.123:0/1575966422 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f72281594a0 0x7f72280c6dd0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 -- 172.21.15.123:0/1575966422 >> 172.21.15.123:0/1575966422 conn(0x7f72280c9d00 msgr2=0x7f722806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 -- 172.21.15.123:0/1575966422 shutdown_connections 2023-10-06T04:32:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.955+0000 7f722fc4d700 1 -- 172.21.15.123:0/1575966422 wait complete. 2023-10-06T04:32:17.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.956+0000 7f722fc4d700 1 Processor -- start 2023-10-06T04:32:17.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.956+0000 7f722fc4d700 1 -- start start 2023-10-06T04:32:17.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722fc4d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7228068d80 0x7f72280c6650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722fc4d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722806e450 0x7f7228169980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722fc4d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72281594a0 0x7f7228169ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722fc4d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72280c6b90 con 0x7f722806e450 2023-10-06T04:32:17.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722fc4d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72281580d0 con 0x7f7228068d80 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722fc4d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7228154660 con 0x7f72281594a0 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722806e450 0x7f7228169980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722806e450 0x7f7228169980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50682/0 (socket says 172.21.15.123:50682) 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 -- 172.21.15.123:0/565681140 learned_addr learned my addr 172.21.15.123:0/565681140 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722e1ea700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72281594a0 0x7f7228169ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 -- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72281594a0 msgr2=0x7f7228169ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d9e9700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7228068d80 0x7f72280c6650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72281594a0 0x7f7228169ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 -- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7228068d80 msgr2=0x7f72280c6650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7228068d80 0x7f72280c6650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 -- 172.21.15.123:0/565681140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72180094f0 con 0x7f722806e450 2023-10-06T04:32:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722e1ea700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72281594a0 0x7f7228169ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.957+0000 7f722d1e8700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722806e450 0x7f7228169980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f722400c600 tx=0x7f7224007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.958+0000 7f721dffb700 1 -- 172.21.15.123:0/565681140 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224011590 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.958+0000 7f721dffb700 1 -- 172.21.15.123:0/565681140 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7224011710 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.958+0000 7f721dffb700 1 -- 172.21.15.123:0/565681140 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7224011ad0 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.958+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72281577c0 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.958+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7228159d00 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.959+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7228158280 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.959+0000 7f721dffb700 1 -- 172.21.15.123:0/565681140 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f722401c460 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.959+0000 7f721dffb700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f720c066f30 0x7f720c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.959+0000 7f721dffb700 1 -- 172.21.15.123:0/565681140 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7224084120 con 0x7f722806e450 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.959+0000 7f722d9e9700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f720c066f30 0x7f720c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.960+0000 7f722d9e9700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f720c066f30 0x7f720c0693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f722806d9e0 tx=0x7f7218016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:17.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:17.964+0000 7f721dffb700 1 -- 172.21.15.123:0/565681140 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7228158280 con 0x7f722806e450 2023-10-06T04:32:18.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.219+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7f7228158280 con 0x7f722806e450 2023-10-06T04:32:18.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.222+0000 7f721dffb700 1 -- 172.21.15.123:0/565681140 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v3) v1 ==== 90+0+0 (secure 0 0 0) 0x7f7228158280 con 0x7f722806e450 2023-10-06T04:32:18.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f720c066f30 msgr2=0x7f720c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f720c066f30 0x7f720c0693e0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f722806d9e0 tx=0x7f7218016000 comp rx=0 tx=0).stop 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722806e450 msgr2=0x7f7228169980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722806e450 0x7f7228169980 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f722400c600 tx=0x7f7224007f60 comp rx=0 tx=0).stop 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 shutdown_connections 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f720c066f30 0x7f720c0693e0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f72281594a0 0x7f7228169ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f7228068d80 0x7f72280c6650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 --2- 172.21.15.123:0/565681140 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f722806e450 0x7f7228169980 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 >> 172.21.15.123:0/565681140 conn(0x7f72280c9d00 msgr2=0x7f7228156430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 shutdown_connections 2023-10-06T04:32:18.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.225+0000 7f722fc4d700 1 -- 172.21.15.123:0/565681140 wait complete. 2023-10-06T04:32:18.238 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-10-06T04:32:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.354+0000 7f9ea5117700 1 Processor -- start 2023-10-06T04:32:18.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.354+0000 7f9ea5117700 1 -- start start 2023-10-06T04:32:18.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9ea5117700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9ea01594a0 0x7f9ea00c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9ea5117700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0068d80 0x7f9ea006e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9ea5117700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ea00cad40 con 0x7f9ea006e450 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9ea5117700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9ea00ca6a0 con 0x7f9ea01594a0 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9ea5117700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00cb550 con 0x7f9ea0068d80 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9e9e59c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9ea006e450 0x7f9ea0068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39354/0 (socket says 172.21.15.123:39354) 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9e9e59c700 1 -- 172.21.15.123:0/2294178954 learned_addr learned my addr 172.21.15.123:0/2294178954 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.355+0000 7f9e9f59e700 1 --2- 172.21.15.123:0/2294178954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0068d80 0x7f9ea006e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9ed9d700 1 --2- 172.21.15.123:0/2294178954 >> v2:172.21.15.177:3300/0 conn(0x7f9ea01594a0 0x7f9ea00c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9dd9b700 1 -- 172.21.15.123:0/2294178954 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 731614752 0 0) 0x7f9ea00cad40 con 0x7f9ea006e450 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9dd9b700 1 -- 172.21.15.123:0/2294178954 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e8c003570 con 0x7f9ea006e450 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9f59e700 1 -- 172.21.15.123:0/2294178954 >> v2:172.21.15.177:3300/0 conn(0x7f9ea01594a0 msgr2=0x7f9ea00c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9f59e700 1 --2- 172.21.15.123:0/2294178954 >> v2:172.21.15.177:3300/0 conn(0x7f9ea01594a0 0x7f9ea00c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9f59e700 1 -- 172.21.15.123:0/2294178954 >> v1:172.21.15.123:6789/0 conn(0x7f9ea006e450 legacy=0x7f9ea0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9f59e700 1 -- 172.21.15.123:0/2294178954 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ea006a920 con 0x7f9ea0068d80 2023-10-06T04:32:18.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9ed9d700 1 --2- 172.21.15.123:0/2294178954 >> v2:172.21.15.177:3300/0 conn(0x7f9ea01594a0 0x7f9ea00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:18.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9f59e700 1 --2- 172.21.15.123:0/2294178954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0068d80 0x7f9ea006e8a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009e60 tx=0x7f9e98009780 comp rx=0 tx=0).ready entity=mon.2 client_cookie=86e80b44bcd9a539 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:18.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9dd9b700 1 -- 172.21.15.123:0/2294178954 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800e070 con 0x7f9ea0068d80 2023-10-06T04:32:18.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9dd9b700 1 -- 172.21.15.123:0/2294178954 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e9800bab0 con 0x7f9ea0068d80 2023-10-06T04:32:18.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.356+0000 7f9e9dd9b700 1 -- 172.21.15.123:0/2294178954 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e9800be30 con 0x7f9ea0068d80 2023-10-06T04:32:18.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 -- 172.21.15.123:0/2294178954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0068d80 msgr2=0x7f9ea006e8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 --2- 172.21.15.123:0/2294178954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0068d80 0x7f9ea006e8a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f9e98009e60 tx=0x7f9e98009780 comp rx=0 tx=0).stop 2023-10-06T04:32:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 -- 172.21.15.123:0/2294178954 shutdown_connections 2023-10-06T04:32:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 --2- 172.21.15.123:0/2294178954 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea0068d80 0x7f9ea006e8a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 --2- 172.21.15.123:0/2294178954 >> v2:172.21.15.177:3300/0 conn(0x7f9ea01594a0 0x7f9ea00c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 -- 172.21.15.123:0/2294178954 >> 172.21.15.123:0/2294178954 conn(0x7f9ea00c9d00 msgr2=0x7f9ea006da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 -- 172.21.15.123:0/2294178954 shutdown_connections 2023-10-06T04:32:18.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.357+0000 7f9ea5117700 1 -- 172.21.15.123:0/2294178954 wait complete. 2023-10-06T04:32:18.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.358+0000 7f9ea5117700 1 Processor -- start 2023-10-06T04:32:18.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.358+0000 7f9ea5117700 1 -- start start 2023-10-06T04:32:18.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9ea5117700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0068d80 0x7f9ea00c6120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9ea5117700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea006e450 0x7f9ea01544e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9ea5117700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea01594a0 0x7f9ea01552a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9ea5117700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00b03e0 con 0x7f9ea01594a0 2023-10-06T04:32:18.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9ea5117700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00c4ce0 con 0x7f9ea0068d80 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9ea5117700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ea00c6660 con 0x7f9ea006e450 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea01594a0 0x7f9ea01552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea01594a0 0x7f9ea01552a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50692/0 (socket says 172.21.15.123:50692) 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 -- 172.21.15.123:0/3003418607 learned_addr learned my addr 172.21.15.123:0/3003418607 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9ed9d700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0068d80 0x7f9ea00c6120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9e59c700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea006e450 0x7f9ea01544e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 -- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea006e450 msgr2=0x7f9ea01544e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea006e450 0x7f9ea01544e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 -- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0068d80 msgr2=0x7f9ea00c6120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0068d80 0x7f9ea00c6120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 -- 172.21.15.123:0/3003418607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e90009420 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e9f59e700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea01594a0 0x7f9ea01552a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00cb690 tx=0x7f9e98019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e96ffd700 1 -- 172.21.15.123:0/3003418607 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98012bc0 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e96ffd700 1 -- 172.21.15.123:0/3003418607 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e98021070 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9e96ffd700 1 -- 172.21.15.123:0/3003418607 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e98026430 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.359+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e98009a20 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.360+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ea00afd90 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.360+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ea00b0590 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.361+0000 7f9e96ffd700 1 -- 172.21.15.123:0/3003418607 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9e98023030 con 0x7f9ea01594a0 2023-10-06T04:32:18.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.361+0000 7f9e96ffd700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9e80066f30 0x7f9e800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.361+0000 7f9e96ffd700 1 -- 172.21.15.123:0/3003418607 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f9e98086300 con 0x7f9ea01594a0 2023-10-06T04:32:18.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.361+0000 7f9e9ed9d700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9e80066f30 0x7f9e800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.362+0000 7f9e9ed9d700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9e80066f30 0x7f9e800693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009f40 tx=0x7f9e900091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:18.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.365+0000 7f9e96ffd700 1 -- 172.21.15.123:0/3003418607 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9ea00b0590 con 0x7f9ea01594a0 2023-10-06T04:32:18.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.625+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f9ea00b0590 con 0x7f9ea01594a0 2023-10-06T04:32:18.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.630+0000 7f9e96ffd700 1 -- 172.21.15.123:0/3003418607 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v4) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9ea00b0590 con 0x7f9ea01594a0 2023-10-06T04:32:18.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9e80066f30 msgr2=0x7f9e800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9e80066f30 0x7f9e800693e0 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f9e90009f40 tx=0x7f9e900091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea01594a0 msgr2=0x7f9ea01552a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea01594a0 0x7f9ea01552a0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9ea00cb690 tx=0x7f9e98019000 comp rx=0 tx=0).stop 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 shutdown_connections 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9e80066f30 0x7f9e800693e0 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ea01594a0 0x7f9ea01552a0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ea0068d80 0x7f9ea00c6120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 --2- 172.21.15.123:0/3003418607 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ea006e450 0x7f9ea01544e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 >> 172.21.15.123:0/3003418607 conn(0x7f9ea00c9d00 msgr2=0x7f9ea00c8340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 shutdown_connections 2023-10-06T04:32:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.632+0000 7f9ea5117700 1 -- 172.21.15.123:0/3003418607 wait complete. 2023-10-06T04:32:18.646 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-10-06T04:32:18.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.762+0000 7f48b11c5700 1 Processor -- start 2023-10-06T04:32:18.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.762+0000 7f48b11c5700 1 -- start start 2023-10-06T04:32:18.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48b11c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac06e450 0x7f48ac06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48b11c5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f48ac068670 0x7f48ac068af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48b11c5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48ac0cad40 con 0x7f48ac1594a0 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48b11c5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48ac0ca6a0 con 0x7f48ac068670 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48b11c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac069030 con 0x7f48ac06e450 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48ab59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f48ac068670 0x7f48ac068af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48ab59e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f48ac068670 0x7f48ac068af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50708/0 (socket says 172.21.15.123:50708) 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48ab59e700 1 -- 172.21.15.123:0/590884732 learned_addr learned my addr 172.21.15.123:0/590884732 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.763+0000 7f48aa59c700 1 --2- 172.21.15.123:0/590884732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac06e450 0x7f48ac06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48ab59e700 1 -- 172.21.15.123:0/590884732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac06e450 msgr2=0x7f48ac06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48ab59e700 1 --2- 172.21.15.123:0/590884732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac06e450 0x7f48ac06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48ab59e700 1 -- 172.21.15.123:0/590884732 >> v1:172.21.15.177:6789/0 conn(0x7f48ac1594a0 legacy=0x7f48ac0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48ab59e700 1 -- 172.21.15.123:0/590884732 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48ac0cb550 con 0x7f48ac068670 2023-10-06T04:32:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48aa59c700 1 --2- 172.21.15.123:0/590884732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac06e450 0x7f48ac06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48ab59e700 1 --2- 172.21.15.123:0/590884732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac068af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009f50 tx=0x7f48a4009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9eb38ce141008c04 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48a9d9b700 1 -- 172.21.15.123:0/590884732 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a400e070 con 0x7f48ac068670 2023-10-06T04:32:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48a9d9b700 1 -- 172.21.15.123:0/590884732 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a400ba70 con 0x7f48ac068670 2023-10-06T04:32:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48a9d9b700 1 -- 172.21.15.123:0/590884732 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a40092e0 con 0x7f48ac068670 2023-10-06T04:32:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 -- 172.21.15.123:0/590884732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 msgr2=0x7f48ac068af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 --2- 172.21.15.123:0/590884732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac068af0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f48a4009f50 tx=0x7f48a4009840 comp rx=0 tx=0).stop 2023-10-06T04:32:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 -- 172.21.15.123:0/590884732 shutdown_connections 2023-10-06T04:32:18.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 --2- 172.21.15.123:0/590884732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac068af0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 --2- 172.21.15.123:0/590884732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac06e450 0x7f48ac06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 -- 172.21.15.123:0/590884732 >> 172.21.15.123:0/590884732 conn(0x7f48ac0c9d00 msgr2=0x7f48ac06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:18.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 -- 172.21.15.123:0/590884732 shutdown_connections 2023-10-06T04:32:18.769 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.764+0000 7f48b11c5700 1 -- 172.21.15.123:0/590884732 wait complete. 2023-10-06T04:32:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.766+0000 7f48b11c5700 1 Processor -- start 2023-10-06T04:32:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.766+0000 7f48b11c5700 1 -- start start 2023-10-06T04:32:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48b11c5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac156230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48b11c5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac06e450 0x7f48ac0c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48b11c5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac1594a0 0x7f48ac0b1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48b11c5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0af6a0 con 0x7f48ac068670 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48b11c5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0af8f0 con 0x7f48ac06e450 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48b11c5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f48ac0afbd0 con 0x7f48ac1594a0 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac156230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac156230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50724/0 (socket says 172.21.15.123:50724) 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 -- 172.21.15.123:0/3226675707 learned_addr learned my addr 172.21.15.123:0/3226675707 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aa59c700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac06e450 0x7f48ac0c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48ab59e700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac1594a0 0x7f48ac0b1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 -- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac1594a0 msgr2=0x7f48ac0b1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac1594a0 0x7f48ac0b1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 -- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac06e450 msgr2=0x7f48ac0c4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac06e450 0x7f48ac0c4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aad9d700 1 -- 172.21.15.123:0/3226675707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48a4009ae0 con 0x7f48ac068670 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.767+0000 7f48aa59c700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac06e450 0x7f48ac0c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.768+0000 7f48aad9d700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac156230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f48a000c4e0 tx=0x7f48a000c8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.768+0000 7f4892ffd700 1 -- 172.21.15.123:0/3226675707 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0011590 con 0x7f48ac068670 2023-10-06T04:32:18.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.768+0000 7f4892ffd700 1 -- 172.21.15.123:0/3226675707 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48a0011710 con 0x7f48ac068670 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.768+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48ac0afeb0 con 0x7f48ac068670 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.768+0000 7f4892ffd700 1 -- 172.21.15.123:0/3226675707 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48a0011ad0 con 0x7f48ac068670 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.768+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48ac156770 con 0x7f48ac068670 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.769+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48ac0b03d0 con 0x7f48ac068670 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.769+0000 7f4892ffd700 1 -- 172.21.15.123:0/3226675707 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f48a0001020 con 0x7f48ac068670 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.769+0000 7f4892ffd700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4898066f30 0x7f48980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.770+0000 7f4892ffd700 1 -- 172.21.15.123:0/3226675707 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f48a0083770 con 0x7f48ac068670 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.770+0000 7f48aa59c700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4898066f30 0x7f48980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.770+0000 7f48aa59c700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4898066f30 0x7f48980693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f489c006f00 tx=0x7f489c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:18.774+0000 7f4892ffd700 1 -- 172.21.15.123:0/3226675707 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f48a0050bd0 con 0x7f48ac068670 2023-10-06T04:32:19.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.029+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f48ac04bb80 con 0x7f48ac068670 2023-10-06T04:32:19.035 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.034+0000 7f4892ffd700 1 -- 172.21.15.123:0/3226675707 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v5) v1 ==== 90+0+0 (secure 0 0 0) 0x7f48a0017020 con 0x7f48ac068670 2023-10-06T04:32:19.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4898066f30 msgr2=0x7f48980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4898066f30 0x7f48980693e0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f489c006f00 tx=0x7f489c008040 comp rx=0 tx=0).stop 2023-10-06T04:32:19.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 msgr2=0x7f48ac156230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac156230 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f48a000c4e0 tx=0x7f48a000c8a0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 shutdown_connections 2023-10-06T04:32:19.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4898066f30 0x7f48980693e0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f48ac068670 0x7f48ac156230 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f48ac1594a0 0x7f48ac0b1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 --2- 172.21.15.123:0/3226675707 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f48ac06e450 0x7f48ac0c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 >> 172.21.15.123:0/3226675707 conn(0x7f48ac0c9d00 msgr2=0x7f48ac0c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:19.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 shutdown_connections 2023-10-06T04:32:19.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.036+0000 7f48b11c5700 1 -- 172.21.15.123:0/3226675707 wait complete. 2023-10-06T04:32:19.050 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-10-06T04:32:19.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.166+0000 7f8a132c1700 1 Processor -- start 2023-10-06T04:32:19.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.166+0000 7f8a132c1700 1 -- start start 2023-10-06T04:32:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a132c1700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8a0c068d80 0x7f8a0c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a132c1700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0cad40 con 0x7f8a0c1594a0 2023-10-06T04:32:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a132c1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a0c0cb550 con 0x7f8a0c06e450 2023-10-06T04:32:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a132c1700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a0c0ca6a0 con 0x7f8a0c068d80 2023-10-06T04:32:19.172 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a1105d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8a0c1594a0 0x7f8a0c0c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39366/0 (socket says 172.21.15.123:39366) 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a1105d700 1 -- 172.21.15.123:0/1886147065 learned_addr learned my addr 172.21.15.123:0/1886147065 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a1185e700 1 --2- 172.21.15.123:0/1886147065 >> v2:172.21.15.177:3300/0 conn(0x7f8a0c068d80 0x7f8a0c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.167+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1804082959 0 0) 0x7f8a0c0cad40 con 0x7f8a0c1594a0 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f89f8003570 con 0x7f8a0c1594a0 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1909225587 0 0) 0x7f89f8003570 con 0x7f8a0c1594a0 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 >> v2:172.21.15.177:3300/0 conn(0x7f8a0c068d80 msgr2=0x7f8a0c06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 --2- 172.21.15.123:0/1886147065 >> v2:172.21.15.177:3300/0 conn(0x7f8a0c068d80 0x7f8a0c06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 >> v1:172.21.15.177:6789/0 conn(0x7f8a0c06e450 legacy=0x7f8a0c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c06a920 con 0x7f8a0c1594a0 2023-10-06T04:32:19.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f89fc001470 con 0x7f8a0c1594a0 2023-10-06T04:32:19.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 --2- 172.21.15.123:0/1886147065 >> v2:172.21.15.177:3300/0 conn(0x7f8a0c068d80 0x7f8a0c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:19.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 >> v1:172.21.15.123:6789/0 conn(0x7f8a0c1594a0 legacy=0x7f8a0c0c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f8003a60 0x7f89f8003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f8004810 0x7f89f8004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f80051c0 0x7f89f800ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f800c0a0 con 0x7f89f80051c0 2023-10-06T04:32:19.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f800c210 con 0x7f89f8004810 2023-10-06T04:32:19.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f800c3d0 con 0x7f89f8003a60 2023-10-06T04:32:19.175 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1105d700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f8003a60 0x7f89f8003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f80051c0 0x7f89f800ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1085c700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f8004810 0x7f89f8004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 -- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f8003a60 msgr2=0x7f89f8003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:19.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f89f8003a60 0x7f89f8003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 -- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f8004810 msgr2=0x7f89f8004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f8004810 0x7f89f8004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 -- 172.21.15.123:0/1886147065 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f8003570 con 0x7f89f80051c0 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 -- 172.21.15.123:0/1886147065 reap_dead start 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.168+0000 7f8a1185e700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f80051c0 0x7f89f800ba60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c069180 tx=0x7f8a0400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1201979285061bf8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a04009c50 con 0x7f89f80051c0 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a0400f040 con 0x7f89f80051c0 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a03fff700 1 -- 172.21.15.123:0/1886147065 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a0400ee90 con 0x7f89f80051c0 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a132c1700 1 -- 172.21.15.123:0/1886147065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f80051c0 msgr2=0x7f89f800ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a132c1700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f80051c0 0x7f89f800ba60 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c069180 tx=0x7f8a0400b040 comp rx=0 tx=0).stop 2023-10-06T04:32:19.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a132c1700 1 -- 172.21.15.123:0/1886147065 shutdown_connections 2023-10-06T04:32:19.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a132c1700 1 --2- 172.21.15.123:0/1886147065 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f89f80051c0 0x7f89f800ba60 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a132c1700 1 -- 172.21.15.123:0/1886147065 >> 172.21.15.123:0/1886147065 conn(0x7f8a0c0c9d00 msgr2=0x7f8a0c06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:19.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a132c1700 1 -- 172.21.15.123:0/1886147065 shutdown_connections 2023-10-06T04:32:19.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.169+0000 7f8a132c1700 1 -- 172.21.15.123:0/1886147065 wait complete. 2023-10-06T04:32:19.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.170+0000 7f8a132c1700 1 Processor -- start 2023-10-06T04:32:19.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.170+0000 7f8a132c1700 1 -- start start 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a132c1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f80051c0 0x7f8a0c145240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a132c1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a0c156470 0x7f8a0c0c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a132c1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a0c1568e0 0x7f8a0c0c72f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a132c1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c153ae0 con 0x7f8a0c156470 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a132c1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c145780 con 0x7f89f80051c0 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a132c1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a0c155130 con 0x7f8a0c1568e0 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a1085c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a0c156470 0x7f8a0c0c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a1085c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a0c156470 0x7f8a0c0c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50742/0 (socket says 172.21.15.123:50742) 2023-10-06T04:32:19.181 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a1085c700 1 -- 172.21.15.123:0/2529879135 learned_addr learned my addr 172.21.15.123:0/2529879135 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a1105d700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f80051c0 0x7f8a0c145240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.171+0000 7f8a1185e700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a0c1568e0 0x7f8a0c0c72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a1085c700 1 -- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a0c1568e0 msgr2=0x7f8a0c0c72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a1085c700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a0c1568e0 0x7f8a0c0c72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a1085c700 1 -- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f80051c0 msgr2=0x7f8a0c145240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a1085c700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f80051c0 0x7f8a0c145240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a1085c700 1 -- 172.21.15.123:0/2529879135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a0c155410 con 0x7f8a0c156470 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a1085c700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a0c156470 0x7f8a0c0c6db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a08002950 tx=0x7f8a08007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a017fa700 1 -- 172.21.15.123:0/2529879135 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a08003390 con 0x7f8a0c156470 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a017fa700 1 -- 172.21.15.123:0/2529879135 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a08013600 con 0x7f8a0c156470 2023-10-06T04:32:19.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a017fa700 1 -- 172.21.15.123:0/2529879135 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a080139a0 con 0x7f8a0c156470 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a0c1556f0 con 0x7f8a0c156470 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.172+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a0c155cb0 con 0x7f8a0c156470 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.173+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a0c063880 con 0x7f8a0c156470 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.174+0000 7f8a017fa700 1 -- 172.21.15.123:0/2529879135 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a08013d50 con 0x7f8a0c156470 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.174+0000 7f8a017fa700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f89f0066f30 0x7f89f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.174+0000 7f8a017fa700 1 -- 172.21.15.123:0/2529879135 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f8a08084540 con 0x7f8a0c156470 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.174+0000 7f8a1105d700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f89f0066f30 0x7f89f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.174+0000 7f8a1105d700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f89f0066f30 0x7f89f00693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0689e0 tx=0x7f89fc001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.178+0000 7f8a017fa700 1 -- 172.21.15.123:0/2529879135 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a08055430 con 0x7f8a0c156470 2023-10-06T04:32:19.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.437+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f8a0c04bb80 con 0x7f8a0c156470 2023-10-06T04:32:19.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.440+0000 7f8a017fa700 1 -- 172.21.15.123:0/2529879135 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v6) v1 ==== 90+0+0 (secure 0 0 0) 0x7f8a08054fc0 con 0x7f8a0c156470 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f89f0066f30 msgr2=0x7f89f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f89f0066f30 0x7f89f00693e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f8a0c0689e0 tx=0x7f89fc001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a0c156470 msgr2=0x7f8a0c0c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a0c156470 0x7f8a0c0c6db0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f8a08002950 tx=0x7f8a08007d30 comp rx=0 tx=0).stop 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 shutdown_connections 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f89f0066f30 0x7f89f00693e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a0c1568e0 0x7f8a0c0c72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f89f80051c0 0x7f8a0c145240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 --2- 172.21.15.123:0/2529879135 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a0c156470 0x7f8a0c0c6db0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 >> 172.21.15.123:0/2529879135 conn(0x7f8a0c0c9d00 msgr2=0x7f8a0c157f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:19.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 shutdown_connections 2023-10-06T04:32:19.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.442+0000 7f8a132c1700 1 -- 172.21.15.123:0/2529879135 wait complete. 2023-10-06T04:32:19.455 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-10-06T04:32:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.572+0000 7fb7ac72d700 1 Processor -- start 2023-10-06T04:32:19.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.572+0000 7fb7ac72d700 1 -- start start 2023-10-06T04:32:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7ac72d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb7a406e960 0x7fb7a406b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.575 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7ac72d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a406ab80 con 0x7fb7a4068e80 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7ac72d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a406ef10 con 0x7fb7a41560e0 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7ac72d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a406f1f0 con 0x7fb7a406e960 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7a9cc8700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb7a4068e80 0x7fb7a406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39380/0 (socket says 172.21.15.123:39380) 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7a9cc8700 1 -- 172.21.15.123:0/483817280 learned_addr learned my addr 172.21.15.123:0/483817280 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7aacca700 1 --2- 172.21.15.123:0/483817280 >> v2:172.21.15.123:3300/0 conn(0x7fb7a406e960 0x7fb7a406b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7aacca700 1 -- 172.21.15.123:0/483817280 >> v1:172.21.15.177:6789/0 conn(0x7fb7a41560e0 legacy=0x7fb7a4068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7aacca700 1 -- 172.21.15.123:0/483817280 >> v1:172.21.15.123:6789/0 conn(0x7fb7a4068e80 legacy=0x7fb7a406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7aacca700 1 -- 172.21.15.123:0/483817280 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a406a530 con 0x7fb7a406e960 2023-10-06T04:32:19.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7a94c7700 1 -- 172.21.15.123:0/483817280 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021254496 0 0) 0x7fb7a406ab80 con 0x7fb7a4068e80 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7aacca700 1 --2- 172.21.15.123:0/483817280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a406e960 0x7fb7a406b010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009e60 tx=0x7fb79c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7707e8da7472e100 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.573+0000 7fb7a94c7700 1 -- 172.21.15.123:0/483817280 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c00e070 con 0x7fb7a406e960 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7a94c7700 1 -- 172.21.15.123:0/483817280 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb79c00bbf0 con 0x7fb7a406e960 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7a94c7700 1 -- 172.21.15.123:0/483817280 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb79c012430 con 0x7fb7a406e960 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7ac72d700 1 -- 172.21.15.123:0/483817280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a406e960 msgr2=0x7fb7a406b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/483817280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a406e960 0x7fb7a406b010 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb79c009e60 tx=0x7fb79c009780 comp rx=0 tx=0).stop 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7ac72d700 1 -- 172.21.15.123:0/483817280 shutdown_connections 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/483817280 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a406e960 0x7fb7a406b010 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7ac72d700 1 -- 172.21.15.123:0/483817280 >> 172.21.15.123:0/483817280 conn(0x7fb7a40cd010 msgr2=0x7fb7a40cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7ac72d700 1 -- 172.21.15.123:0/483817280 shutdown_connections 2023-10-06T04:32:19.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.574+0000 7fb7ac72d700 1 -- 172.21.15.123:0/483817280 wait complete. 2023-10-06T04:32:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.575+0000 7fb7ac72d700 1 Processor -- start 2023-10-06T04:32:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.575+0000 7fb7ac72d700 1 -- start start 2023-10-06T04:32:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7ac72d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a4068e80 0x7fb7a40b2740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7ac72d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a406e960 0x7fb7a41545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7ac72d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a41560e0 0x7fb7a4158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7ac72d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b1f40 con 0x7fb7a4068e80 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7ac72d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a41565e0 con 0x7fb7a406e960 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7ac72d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a40b2c80 con 0x7fb7a41560e0 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a4068e80 0x7fb7a40b2740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a4068e80 0x7fb7a40b2740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50754/0 (socket says 172.21.15.123:50754) 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 -- 172.21.15.123:0/4208848896 learned_addr learned my addr 172.21.15.123:0/4208848896 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aacca700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a41560e0 0x7fb7a4158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 -- 172.21.15.123:0/4208848896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a41560e0 msgr2=0x7fb7a4158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a41560e0 0x7fb7a4158890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 -- 172.21.15.123:0/4208848896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a406e960 msgr2=0x7fb7a41545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a406e960 0x7fb7a41545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.576+0000 7fb7aa4c9700 1 -- 172.21.15.123:0/4208848896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb79c009a20 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.577+0000 7fb7aa4c9700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a4068e80 0x7fb7a40b2740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b5e0 tx=0x7fb7a0009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.577+0000 7fb79a7fc700 1 -- 172.21.15.123:0/4208848896 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000f510 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.577+0000 7fb79a7fc700 1 -- 172.21.15.123:0/4208848896 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7a000f670 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.577+0000 7fb79a7fc700 1 -- 172.21.15.123:0/4208848896 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7a000fa30 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.577+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a40ac270 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.577+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a40b1c60 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.578+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a406d2b0 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.578+0000 7fb79a7fc700 1 -- 172.21.15.123:0/4208848896 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb7a001a400 con 0x7fb7a4068e80 2023-10-06T04:32:19.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.578+0000 7fb79a7fc700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb788066f30 0x7fb7880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.579+0000 7fb79a7fc700 1 -- 172.21.15.123:0/4208848896 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fb7a0083ec0 con 0x7fb7a4068e80 2023-10-06T04:32:19.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.579+0000 7fb7a9cc8700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb788066f30 0x7fb7880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.579+0000 7fb7a9cc8700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb788066f30 0x7fb7880693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7940011e0 tx=0x7fb7940079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.583+0000 7fb79a7fc700 1 -- 172.21.15.123:0/4208848896 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb7a00513a0 con 0x7fb7a4068e80 2023-10-06T04:32:19.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.837+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7fb7a404bb80 con 0x7fb7a4068e80 2023-10-06T04:32:19.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.841+0000 7fb79a7fc700 1 -- 172.21.15.123:0/4208848896 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v7) v1 ==== 97+0+0 (secure 0 0 0) 0x7fb7a00549c0 con 0x7fb7a4068e80 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb788066f30 msgr2=0x7fb7880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb788066f30 0x7fb7880693e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb7940011e0 tx=0x7fb7940079d0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a4068e80 msgr2=0x7fb7a40b2740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a4068e80 0x7fb7a40b2740 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb7a000b5e0 tx=0x7fb7a0009120 comp rx=0 tx=0).stop 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 shutdown_connections 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb788066f30 0x7fb7880693e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb7a4068e80 0x7fb7a40b2740 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb7a41560e0 0x7fb7a4158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 --2- 172.21.15.123:0/4208848896 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb7a406e960 0x7fb7a41545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.845 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 >> 172.21.15.123:0/4208848896 conn(0x7fb7a40cd010 msgr2=0x7fb7a4068700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:19.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 shutdown_connections 2023-10-06T04:32:19.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.843+0000 7fb7ac72d700 1 -- 172.21.15.123:0/4208848896 wait complete. 2023-10-06T04:32:19.856 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-10-06T04:32:19.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.973+0000 7fedcc43a700 1 Processor -- start 2023-10-06T04:32:19.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.973+0000 7fedcc43a700 1 -- start start 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedcc43a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 0x7fedc406e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedcc43a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40cad40 con 0x7fedc4068670 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedcc43a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedc40cb550 con 0x7fedc41594a0 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedcc43a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40ca6a0 con 0x7fedc406e450 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedca9d7700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fedc4068670 0x7fedc406adb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39388/0 (socket says 172.21.15.123:39388) 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedca9d7700 1 -- 172.21.15.123:0/2375550942 learned_addr learned my addr 172.21.15.123:0/2375550942 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc99d5700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 0x7fedc406e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1247270938 0 0) 0x7fedc40cad40 con 0x7fedc4068670 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedb0003570 con 0x7fedc4068670 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3697249290 0 0) 0x7fedb0003570 con 0x7fedc4068670 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 msgr2=0x7fedc406e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 0x7fedc406e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 >> v1:172.21.15.177:6789/0 conn(0x7fedc41594a0 legacy=0x7fedc40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc406a920 con 0x7fedc4068670 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.974+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fedbc0017f0 con 0x7fedc4068670 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 >> v1:172.21.15.123:6789/0 conn(0x7fedc4068670 legacy=0x7fedc406adb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0003a90 0x7fedb0003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 0x7fedb0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedb0004d50 0x7fedb00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb000bb10 con 0x7fedb0004d50 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb000bc50 con 0x7fedb0003a90 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb000be10 con 0x7fedc406e450 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca9d7700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedb0004d50 0x7fedb00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca1d6700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0003a90 0x7fedb0003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc99d5700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 0x7fedb0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca9d7700 1 -- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 msgr2=0x7fedb0004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca9d7700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc406e450 0x7fedb0004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca9d7700 1 -- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0003a90 msgr2=0x7fedb0003e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca9d7700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0003a90 0x7fedb0003e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca9d7700 1 -- 172.21.15.123:0/2375550942 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb0003570 con 0x7fedb0004d50 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedca9d7700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedb0004d50 0x7fedb00051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00a910 tx=0x7fedbc002170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f7105a2ecacf03 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc015b00 con 0x7fedb0004d50 2023-10-06T04:32:19.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedbc015c60 con 0x7fedb0004d50 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.975+0000 7fedc91d4700 1 -- 172.21.15.123:0/2375550942 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedbc01e470 con 0x7fedb0004d50 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.976+0000 7fedcc43a700 1 -- 172.21.15.123:0/2375550942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedb0004d50 msgr2=0x7fedb00051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.976+0000 7fedcc43a700 1 --2- 172.21.15.123:0/2375550942 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedb0004d50 0x7fedb00051c0 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fedbc00a910 tx=0x7fedbc002170 comp rx=0 tx=0).stop 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.976+0000 7fedca9d7700 1 -- 172.21.15.123:0/2375550942 reap_dead start 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.976+0000 7fedcc43a700 1 -- 172.21.15.123:0/2375550942 shutdown_connections 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.976+0000 7fedcc43a700 1 -- 172.21.15.123:0/2375550942 >> 172.21.15.123:0/2375550942 conn(0x7fedc40c9d00 msgr2=0x7fedc406da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.976+0000 7fedcc43a700 1 -- 172.21.15.123:0/2375550942 shutdown_connections 2023-10-06T04:32:19.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.976+0000 7fedcc43a700 1 -- 172.21.15.123:0/2375550942 wait complete. 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.977+0000 7fedcc43a700 1 Processor -- start 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.977+0000 7fedcc43a700 1 -- start start 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedcc43a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0004d50 0x7fedc40b1050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedcc43a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc4151140 0x7fedc4168020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedcc43a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc4168560 0x7fedc41689d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedcc43a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc41540c0 con 0x7fedc4151140 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedcc43a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40b1fb0 con 0x7fedb0004d50 2023-10-06T04:32:19.985 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedcc43a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedc40c4130 con 0x7fedc4168560 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc4151140 0x7fedc4168020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc4151140 0x7fedc4168020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:50766/0 (socket says 172.21.15.123:50766) 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 -- 172.21.15.123:0/3455228503 learned_addr learned my addr 172.21.15.123:0/3455228503 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedca9d7700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc4168560 0x7fedc41689d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedca1d6700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0004d50 0x7fedc40b1050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 -- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc4168560 msgr2=0x7fedc41689d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc4168560 0x7fedc41689d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 -- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0004d50 msgr2=0x7fedc40b1050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0004d50 0x7fedc40b1050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 -- 172.21.15.123:0/3455228503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedc4168f10 con 0x7fedc4151140 2023-10-06T04:32:19.986 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedca1d6700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0004d50 0x7fedc40b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.978+0000 7fedc99d5700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc4151140 0x7fedc4168020 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fedc0011b40 tx=0x7fedc00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.979+0000 7fedba7fc700 1 -- 172.21.15.123:0/3455228503 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc000fa10 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.979+0000 7fedba7fc700 1 -- 172.21.15.123:0/3455228503 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedc000fb70 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.979+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedc40b1710 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.979+0000 7fedba7fc700 1 -- 172.21.15.123:0/3455228503 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedc0014440 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.979+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedc40b1cd0 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.980+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedc406c9b0 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.980+0000 7fedba7fc700 1 -- 172.21.15.123:0/3455228503 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fedc0014870 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.980+0000 7fedba7fc700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feda8066f30 0x7feda80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.980+0000 7fedba7fc700 1 -- 172.21.15.123:0/3455228503 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fedc0018030 con 0x7fedc4151140 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.981+0000 7fedca1d6700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feda8066f30 0x7feda80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:19.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.981+0000 7fedca1d6700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feda8066f30 0x7feda80693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fedb4003c50 tx=0x7fedb4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:19.988 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:19.985+0000 7fedba7fc700 1 -- 172.21.15.123:0/3455228503 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fedc00521f0 con 0x7fedc4151140 2023-10-06T04:32:20.242 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.241+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7fedc40c4270 con 0x7fedc4151140 2023-10-06T04:32:20.246 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.245+0000 7fedba7fc700 1 -- 172.21.15.123:0/3455228503 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7fedc00220a0 con 0x7fedc4151140 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feda8066f30 msgr2=0x7feda80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feda8066f30 0x7feda80693e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fedb4003c50 tx=0x7fedb4009040 comp rx=0 tx=0).stop 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc4151140 msgr2=0x7fedc4168020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc4151140 0x7fedc4168020 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fedc0011b40 tx=0x7fedc00091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 shutdown_connections 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feda8066f30 0x7feda80693e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fedc4168560 0x7fedc41689d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fedb0004d50 0x7fedc40b1050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 --2- 172.21.15.123:0/3455228503 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fedc4151140 0x7fedc4168020 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 >> 172.21.15.123:0/3455228503 conn(0x7fedc40c9d00 msgr2=0x7fedc406adb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:20.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 shutdown_connections 2023-10-06T04:32:20.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.247+0000 7fedcc43a700 1 -- 172.21.15.123:0/3455228503 wait complete. 2023-10-06T04:32:20.261 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.376+0000 7fc9e3250700 1 Processor -- start 2023-10-06T04:32:20.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.376+0000 7fc9e3250700 1 -- start start 2023-10-06T04:32:20.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e3250700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc9dc067660 0x7fc9dc067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e3250700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc0c9570 0x7fc9dc06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e3250700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9dc067f70 con 0x7fc9dc0ca5b0 2023-10-06T04:32:20.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e3250700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9dc06a750 con 0x7fc9dc067660 2023-10-06T04:32:20.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e3250700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc06aa30 con 0x7fc9dc0c9570 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e17ed700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc9dc0ca5b0 0x7fc9dc06e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40296/0 (socket says 172.21.15.123:40296) 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e0fec700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc9dc067660 0x7fc9dc067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.377+0000 7fc9e17ed700 1 -- 172.21.15.123:0/526800701 learned_addr learned my addr 172.21.15.123:0/526800701 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9dbfff700 1 --2- 172.21.15.123:0/526800701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc0c9570 0x7fc9dc06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9db7fe700 1 -- 172.21.15.123:0/526800701 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1782772601 0 0) 0x7fc9dc067f70 con 0x7fc9dc0ca5b0 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9db7fe700 1 -- 172.21.15.123:0/526800701 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9cc003570 con 0x7fc9dc0ca5b0 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9e0fec700 1 -- 172.21.15.123:0/526800701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc0c9570 msgr2=0x7fc9dc06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9e0fec700 1 --2- 172.21.15.123:0/526800701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc0c9570 0x7fc9dc06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9e0fec700 1 -- 172.21.15.123:0/526800701 >> v1:172.21.15.123:6789/0 conn(0x7fc9dc0ca5b0 legacy=0x7fc9dc06e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9e0fec700 1 -- 172.21.15.123:0/526800701 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9dc06ad10 con 0x7fc9dc067660 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9e0fec700 1 --2- 172.21.15.123:0/526800701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc067660 0x7fc9dc067a30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00098e0 tx=0x7fc9d00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e0d55314273b7c2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:20.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9db7fe700 1 -- 172.21.15.123:0/526800701 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0011630 con 0x7fc9dc067660 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9db7fe700 1 -- 172.21.15.123:0/526800701 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9d0011790 con 0x7fc9dc067660 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.378+0000 7fc9db7fe700 1 -- 172.21.15.123:0/526800701 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9d0011b30 con 0x7fc9dc067660 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 -- 172.21.15.123:0/526800701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc067660 msgr2=0x7fc9dc067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 --2- 172.21.15.123:0/526800701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc067660 0x7fc9dc067a30 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00098e0 tx=0x7fc9d00091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 -- 172.21.15.123:0/526800701 shutdown_connections 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 --2- 172.21.15.123:0/526800701 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc067660 0x7fc9dc067a30 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 --2- 172.21.15.123:0/526800701 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc0c9570 0x7fc9dc06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 -- 172.21.15.123:0/526800701 >> 172.21.15.123:0/526800701 conn(0x7fc9dc0cda80 msgr2=0x7fc9dc0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 -- 172.21.15.123:0/526800701 shutdown_connections 2023-10-06T04:32:20.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.379+0000 7fc9e3250700 1 -- 172.21.15.123:0/526800701 wait complete. 2023-10-06T04:32:20.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.380+0000 7fc9e3250700 1 Processor -- start 2023-10-06T04:32:20.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.380+0000 7fc9e3250700 1 -- start start 2023-10-06T04:32:20.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9e3250700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc067660 0x7fc9dc15b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9e3250700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc0c9570 0x7fc9dc159b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9e3250700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9dc0ca5b0 0x7fc9dc156c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9e3250700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0c0a50 con 0x7fc9dc0c9570 2023-10-06T04:32:20.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9e3250700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0aec90 con 0x7fc9dc0ca5b0 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9e3250700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9dc0aef70 con 0x7fc9dc067660 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc0c9570 0x7fc9dc159b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9dbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc0c9570 0x7fc9dc159b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35608/0 (socket says 172.21.15.123:35608) 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.381+0000 7fc9dbfff700 1 -- 172.21.15.123:0/3026220019 learned_addr learned my addr 172.21.15.123:0/3026220019 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9e0fec700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc067660 0x7fc9dc15b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9dbfff700 1 -- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc067660 msgr2=0x7fc9dc15b8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9e17ed700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9dc0ca5b0 0x7fc9dc156c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9dbfff700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc067660 0x7fc9dc15b8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9dbfff700 1 -- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9dc0ca5b0 msgr2=0x7fc9dc156c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9dbfff700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9dc0ca5b0 0x7fc9dc156c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9dbfff700 1 -- 172.21.15.123:0/3026220019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d0009420 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9e17ed700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9dc0ca5b0 0x7fc9dc156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9dbfff700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc0c9570 0x7fc9dc159b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800e900 tx=0x7fc9c80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9d8ff9700 1 -- 172.21.15.123:0/3026220019 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c800eea0 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9d8ff9700 1 -- 172.21.15.123:0/3026220019 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c800c850 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9dc0af250 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9d8ff9700 1 -- 172.21.15.123:0/3026220019 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c800cc10 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.382+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9dc0ad410 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.383+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9dc0cde80 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.383+0000 7fc9d8ff9700 1 -- 172.21.15.123:0/3026220019 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc9c8013470 con 0x7fc9dc0c9570 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.384+0000 7fc9d8ff9700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc9c0066fb0 0x7fc9c0069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.384+0000 7fc9d8ff9700 1 -- 172.21.15.123:0/3026220019 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7fc9c8084550 con 0x7fc9dc0c9570 2023-10-06T04:32:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.384+0000 7fc9e0fec700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc9c0066fb0 0x7fc9c0069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.384+0000 7fc9e0fec700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc9c0066fb0 0x7fc9c0069460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00091e0 tx=0x7fc9d0017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:20.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.388+0000 7fc9d8ff9700 1 -- 172.21.15.123:0/3026220019 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc9c8051910 con 0x7fc9dc0c9570 2023-10-06T04:32:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.571+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9dc067ee0 con 0x7fc9dc0c9570 2023-10-06T04:32:20.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.572+0000 7fc9d8ff9700 1 -- 172.21.15.123:0/3026220019 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v30) v1 ==== 74+0+13794 (secure 0 0 0) 0x7fc9c8054f30 con 0x7fc9dc0c9570 2023-10-06T04:32:20.574 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:20.574 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":30,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:17.409580+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":25,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":21,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:20.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.574+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc9c0066fb0 msgr2=0x7fc9c0069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.574+0000 7fc9e3250700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc9c0066fb0 0x7fc9c0069460 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc9d00091e0 tx=0x7fc9d0017000 comp rx=0 tx=0).stop 2023-10-06T04:32:20.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.574+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc0c9570 msgr2=0x7fc9dc159b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.574+0000 7fc9e3250700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc0c9570 0x7fc9dc159b40 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7fc9c800e900 tx=0x7fc9c80091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 shutdown_connections 2023-10-06T04:32:20.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc9c0066fb0 0x7fc9c0069460 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc9dc0ca5b0 0x7fc9dc156c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc9dc067660 0x7fc9dc15b8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 --2- 172.21.15.123:0/3026220019 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc9dc0c9570 0x7fc9dc159b40 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 >> 172.21.15.123:0/3026220019 conn(0x7fc9dc0cda80 msgr2=0x7fc9dc0a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:20.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 shutdown_connections 2023-10-06T04:32:20.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.575+0000 7fc9e3250700 1 -- 172.21.15.123:0/3026220019 wait complete. 2023-10-06T04:32:20.588 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}] 2023-10-06T04:32:20.588 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:20.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.707+0000 7efd870a7700 1 Processor -- start 2023-10-06T04:32:20.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.707+0000 7efd870a7700 1 -- start start 2023-10-06T04:32:20.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd870a7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 0x7efd800a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd870a7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd80069aa0 con 0x7efd800c8710 2023-10-06T04:32:20.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd870a7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efd8006a340 con 0x7efd80068d80 2023-10-06T04:32:20.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd870a7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd800aa6f0 con 0x7efd800c8340 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7ffff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efd800c8710 0x7efd80068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40302/0 (socket says 172.21.15.123:40302) 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7ffff700 1 -- 172.21.15.123:0/4238704798 learned_addr learned my addr 172.21.15.123:0/4238704798 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd84e43700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 0x7efd800a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1270707343 0 0) 0x7efd80069aa0 con 0x7efd800c8710 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efd70003570 con 0x7efd800c8710 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695968753 0 0) 0x7efd70003570 con 0x7efd800c8710 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 msgr2=0x7efd800a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7f7fe700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 0x7efd800a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.708+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 >> v1:172.21.15.177:6789/0 conn(0x7efd80068d80 legacy=0x7efd8006a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd80069cf0 con 0x7efd800c8710 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7efd6c001470 con 0x7efd800c8710 2023-10-06T04:32:20.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 >> v1:172.21.15.123:6789/0 conn(0x7efd800c8710 legacy=0x7efd80068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd70003e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd70004810 0x7efd70004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 0x7efd700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800aa6f0 con 0x7efd70003a90 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd7000bb10 con 0x7efd70004810 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd7000bcd0 con 0x7efd800c8340 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd84e43700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd70003e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7ffff700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd70004810 0x7efd70004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd85644700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 0x7efd700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd84e43700 1 -- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 msgr2=0x7efd700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:20.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd84e43700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800c8340 0x7efd700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd84e43700 1 -- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd70004810 msgr2=0x7efd70004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd84e43700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd70004810 0x7efd70004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd84e43700 1 -- 172.21.15.123:0/4238704798 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd70003570 con 0x7efd70003a90 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd84e43700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd70003e60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efd74009c00 tx=0x7efd740091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e9ee8996af68c67 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400e710 con 0x7efd70003a90 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7400e870 con 0x7efd70003a90 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.709+0000 7efd7f7fe700 1 -- 172.21.15.123:0/4238704798 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd7400ec30 con 0x7efd70003a90 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.710+0000 7efd870a7700 1 -- 172.21.15.123:0/4238704798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 msgr2=0x7efd70003e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.710+0000 7efd870a7700 1 --2- 172.21.15.123:0/4238704798 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd70003e60 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7efd74009c00 tx=0x7efd740091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.710+0000 7efd85644700 1 -- 172.21.15.123:0/4238704798 reap_dead start 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.710+0000 7efd870a7700 1 -- 172.21.15.123:0/4238704798 shutdown_connections 2023-10-06T04:32:20.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.710+0000 7efd870a7700 1 -- 172.21.15.123:0/4238704798 >> 172.21.15.123:0/4238704798 conn(0x7efd800c9ca0 msgr2=0x7efd8006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:20.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.710+0000 7efd870a7700 1 -- 172.21.15.123:0/4238704798 shutdown_connections 2023-10-06T04:32:20.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.710+0000 7efd870a7700 1 -- 172.21.15.123:0/4238704798 wait complete. 2023-10-06T04:32:20.716 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.711+0000 7efd870a7700 1 Processor -- start 2023-10-06T04:32:20.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd870a7700 1 -- start start 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd870a7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd800b1310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd870a7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800af630 0x7efd8015e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd870a7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd8015b4b0 0x7efd8015b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd870a7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800b06b0 con 0x7efd70003a90 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd870a7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efd800c4190 con 0x7efd8015b4b0 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd870a7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efd8015be40 con 0x7efd800af630 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd84e43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd800b1310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd84e43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd800b1310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35616/0 (socket says 172.21.15.123:35616) 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.712+0000 7efd84e43700 1 -- 172.21.15.123:0/3253779975 learned_addr learned my addr 172.21.15.123:0/3253779975 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd7ffff700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800af630 0x7efd8015e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.719 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd85644700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd8015b4b0 0x7efd8015b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd84e43700 1 -- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800af630 msgr2=0x7efd8015e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd84e43700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800af630 0x7efd8015e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd84e43700 1 -- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd8015b4b0 msgr2=0x7efd8015b900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd84e43700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd8015b4b0 0x7efd8015b900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd84e43700 1 -- 172.21.15.123:0/3253779975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efd800afe10 con 0x7efd70003a90 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd85644700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd8015b4b0 0x7efd8015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd84e43700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd800b1310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efd7400e4b0 tx=0x7efd7401a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd7cff9700 1 -- 172.21.15.123:0/3253779975 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74022070 con 0x7efd70003a90 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd7cff9700 1 -- 172.21.15.123:0/3253779975 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efd7400e9e0 con 0x7efd70003a90 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efd800b00f0 con 0x7efd70003a90 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd7cff9700 1 -- 172.21.15.123:0/3253779975 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efd74027470 con 0x7efd70003a90 2023-10-06T04:32:20.720 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.713+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efd800c43c0 con 0x7efd70003a90 2023-10-06T04:32:20.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.714+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efd800c45b0 con 0x7efd70003a90 2023-10-06T04:32:20.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.714+0000 7efd7cff9700 1 -- 172.21.15.123:0/3253779975 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efd7401c450 con 0x7efd70003a90 2023-10-06T04:32:20.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.715+0000 7efd7cff9700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd64066f60 0x7efd64069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:20.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.715+0000 7efd7cff9700 1 -- 172.21.15.123:0/3253779975 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7efd74085e20 con 0x7efd70003a90 2023-10-06T04:32:20.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.715+0000 7efd7ffff700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd64066f60 0x7efd64069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:20.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.715+0000 7efd7ffff700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd64066f60 0x7efd64069410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd6c001120 tx=0x7efd6c007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:20.721 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.719+0000 7efd7cff9700 1 -- 172.21.15.123:0/3253779975 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efd740532c0 con 0x7efd70003a90 2023-10-06T04:32:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.898+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7efd80069fd0 con 0x7efd70003a90 2023-10-06T04:32:20.900 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.899+0000 7efd7cff9700 1 -- 172.21.15.123:0/3253779975 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v30) v1 ==== 93+0+10 (secure 0 0 0) 0x7efd740170a0 con 0x7efd70003a90 2023-10-06T04:32:20.900 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd64066f60 msgr2=0x7efd64069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd64066f60 0x7efd64069410 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7efd6c001120 tx=0x7efd6c007040 comp rx=0 tx=0).stop 2023-10-06T04:32:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 msgr2=0x7efd800b1310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:20.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd800b1310 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7efd7400e4b0 tx=0x7efd7401a000 comp rx=0 tx=0).stop 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 shutdown_connections 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efd64066f60 0x7efd64069410 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efd70003a90 0x7efd800b1310 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efd8015b4b0 0x7efd8015b900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 --2- 172.21.15.123:0/3253779975 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efd800af630 0x7efd8015e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 >> 172.21.15.123:0/3253779975 conn(0x7efd800c9ca0 msgr2=0x7efd800b1df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 shutdown_connections 2023-10-06T04:32:20.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:20.901+0000 7efd870a7700 1 -- 172.21.15.123:0/3253779975 wait complete. 2023-10-06T04:32:20.914 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:21.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.033+0000 7f37aeccc700 1 Processor -- start 2023-10-06T04:32:21.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.033+0000 7f37aeccc700 1 -- start start 2023-10-06T04:32:21.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37aeccc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 0x7f37a8068770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37aeccc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a80c9640 con 0x7f37a80cab40 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37aeccc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37a80c9030 con 0x7f37a80cc930 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37aeccc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80cd530 con 0x7f37a81597d0 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a7fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f37a80cab40 0x7f37a80cc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40316/0 (socket says 172.21.15.123:40316) 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a7fff700 1 -- 172.21.15.123:0/2423710891 learned_addr learned my addr 172.21.15.123:0/2423710891 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37aca68700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 0x7f37a8068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3392828922 0 0) 0x7f37a80c9640 con 0x7f37a80cab40 2023-10-06T04:32:21.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3798003570 con 0x7f37a80cab40 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4185821931 0 0) 0x7f3798003570 con 0x7f37a80cab40 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 msgr2=0x7f37a8068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 0x7f37a8068770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 >> v1:172.21.15.177:6789/0 conn(0x7f37a80cc930 legacy=0x7f37a80c9ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a80ccf70 con 0x7f37a80cab40 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f3794001470 con 0x7f37a80cab40 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.034+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 >> v1:172.21.15.123:6789/0 conn(0x7f37a80cab40 legacy=0x7f37a80cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798003a90 0x7f3798003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3798004810 0x7f3798004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 0x7f37980051e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80cd530 con 0x7f3798004810 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f379800bb60 con 0x7f3798003a90 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f379800bd20 con 0x7f37a81597d0 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a7fff700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3798004810 0x7f3798004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37aca68700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798003a90 0x7f3798003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37ad269700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 0x7f37980051e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a7fff700 1 -- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 msgr2=0x7f37980051e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a7fff700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 0x7f37980051e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a7fff700 1 -- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798003a90 msgr2=0x7f3798003ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:21.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a7fff700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798003a90 0x7f3798003ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a7fff700 1 -- 172.21.15.123:0/2423710891 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3798003570 con 0x7f3798004810 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37ad269700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a81597d0 0x7f37980051e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a7fff700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3798004810 0x7f3798004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f379400c3c0 tx=0x7f3794001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba24e8f6b1e48887 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f379400cc90 con 0x7f3798004810 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f379400cdf0 con 0x7f3798004810 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.035+0000 7f37a77fe700 1 -- 172.21.15.123:0/2423710891 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3794005b50 con 0x7f3798004810 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.036+0000 7f37aeccc700 1 -- 172.21.15.123:0/2423710891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3798004810 msgr2=0x7f3798004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.036+0000 7f37aeccc700 1 --2- 172.21.15.123:0/2423710891 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3798004810 0x7f3798004c80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f379400c3c0 tx=0x7f3794001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.036+0000 7f37ad269700 1 -- 172.21.15.123:0/2423710891 reap_dead start 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.036+0000 7f37aeccc700 1 -- 172.21.15.123:0/2423710891 shutdown_connections 2023-10-06T04:32:21.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.036+0000 7f37aeccc700 1 -- 172.21.15.123:0/2423710891 >> 172.21.15.123:0/2423710891 conn(0x7f37a8097450 msgr2=0x7f37a8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.036+0000 7f37aeccc700 1 -- 172.21.15.123:0/2423710891 shutdown_connections 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.036+0000 7f37aeccc700 1 -- 172.21.15.123:0/2423710891 wait complete. 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.037+0000 7f37aeccc700 1 Processor -- start 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.037+0000 7f37aeccc700 1 -- start start 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37aeccc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798004810 0x7f37a816b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37aeccc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a80c3220 0x7f37a814e4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37aeccc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8153310 0x7f37a81537b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37aeccc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80ccd90 con 0x7f37a8153310 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37aeccc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37a8153cf0 con 0x7f3798004810 2023-10-06T04:32:21.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37aeccc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37a80b1ae0 con 0x7f37a80c3220 2023-10-06T04:32:21.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8153310 0x7f37a81537b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8153310 0x7f37a81537b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35630/0 (socket says 172.21.15.123:35630) 2023-10-06T04:32:21.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 -- 172.21.15.123:0/2331212660 learned_addr learned my addr 172.21.15.123:0/2331212660 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:21.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37a7fff700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a80c3220 0x7f37a814e4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37aca68700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798004810 0x7f37a816b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 -- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a80c3220 msgr2=0x7f37a814e4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a80c3220 0x7f37a814e4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 -- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798004810 msgr2=0x7f37a816b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798004810 0x7f37a816b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 -- 172.21.15.123:0/2331212660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37a816b490 con 0x7f37a8153310 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37ad269700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8153310 0x7f37a81537b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f37a806e1c0 tx=0x7f37a0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.038+0000 7f37a4ff9700 1 -- 172.21.15.123:0/2331212660 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0013630 con 0x7f37a8153310 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.039+0000 7f37a4ff9700 1 -- 172.21.15.123:0/2331212660 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37a0013790 con 0x7f37a8153310 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.039+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37a816a910 con 0x7f37a8153310 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.039+0000 7f37a4ff9700 1 -- 172.21.15.123:0/2331212660 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37a0013b50 con 0x7f37a8153310 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.039+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37a816aed0 con 0x7f37a8153310 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.040+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37a80c39e0 con 0x7f37a8153310 2023-10-06T04:32:21.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.040+0000 7f37a4ff9700 1 -- 172.21.15.123:0/2331212660 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f37a0003030 con 0x7f37a8153310 2023-10-06T04:32:21.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.040+0000 7f37a4ff9700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f378c066f30 0x7f378c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.040+0000 7f37a4ff9700 1 -- 172.21.15.123:0/2331212660 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f37a00842a0 con 0x7f37a8153310 2023-10-06T04:32:21.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.040+0000 7f37aca68700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f378c066f30 0x7f378c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.041+0000 7f37aca68700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f378c066f30 0x7f378c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f379c006aa0 tx=0x7f379c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.044+0000 7f37a4ff9700 1 -- 172.21.15.123:0/2331212660 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37a80c39e0 con 0x7f37a8153310 2023-10-06T04:32:21.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.225+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f37a80c39e0 con 0x7f37a8153310 2023-10-06T04:32:21.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.225+0000 7f37a4ff9700 1 -- 172.21.15.123:0/2331212660 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v30) v1 ==== 92+0+10 (secure 0 0 0) 0x7f37a80c39e0 con 0x7f37a8153310 2023-10-06T04:32:21.227 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:21.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f378c066f30 msgr2=0x7f378c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f378c066f30 0x7f378c0693e0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f379c006aa0 tx=0x7f379c008040 comp rx=0 tx=0).stop 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8153310 msgr2=0x7f37a81537b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8153310 0x7f37a81537b0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f37a806e1c0 tx=0x7f37a0009040 comp rx=0 tx=0).stop 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 shutdown_connections 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f378c066f30 0x7f378c0693e0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f37a8153310 0x7f37a81537b0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3798004810 0x7f37a816b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 --2- 172.21.15.123:0/2331212660 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f37a80c3220 0x7f37a814e4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 >> 172.21.15.123:0/2331212660 conn(0x7f37a8097450 msgr2=0x7f37a81552d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:21.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 shutdown_connections 2023-10-06T04:32:21.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.227+0000 7f37aeccc700 1 -- 172.21.15.123:0/2331212660 wait complete. 2023-10-06T04:32:21.240 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-10-06T04:32:21.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.356+0000 7f7353a51700 1 Processor -- start 2023-10-06T04:32:21.358 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7353a51700 1 -- start start 2023-10-06T04:32:21.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7353a51700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f734c067660 0x7f734c067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7353a51700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c0c9570 0x7f734c06e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7353a51700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f734c067f70 con 0x7f734c0ca5b0 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7353a51700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f734c06a750 con 0x7f734c067660 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7353a51700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c06aa30 con 0x7f734c0c9570 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f73517ed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f734c067660 0x7f734c067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7351fee700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f734c0ca5b0 0x7f734c06e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40320/0 (socket says 172.21.15.123:40320) 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.357+0000 7f7351fee700 1 -- 172.21.15.123:0/2805817708 learned_addr learned my addr 172.21.15.123:0/2805817708 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f7350fec700 1 --2- 172.21.15.123:0/2805817708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c0c9570 0x7f734c06e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f73517ed700 1 -- 172.21.15.123:0/2805817708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c0c9570 msgr2=0x7f734c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f73517ed700 1 --2- 172.21.15.123:0/2805817708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c0c9570 0x7f734c06e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f73517ed700 1 -- 172.21.15.123:0/2805817708 >> v1:172.21.15.123:6789/0 conn(0x7f734c0ca5b0 legacy=0x7f734c06e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f73517ed700 1 -- 172.21.15.123:0/2805817708 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f734c06ad10 con 0x7f734c067660 2023-10-06T04:32:21.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f733ffff700 1 -- 172.21.15.123:0/2805817708 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 755458819 0 0) 0x7f734c067f70 con 0x7f734c0ca5b0 2023-10-06T04:32:21.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f73517ed700 1 --2- 172.21.15.123:0/2805817708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c067660 0x7f734c067a30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f73400098e0 tx=0x7f73400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e65419d59da59ec server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f733ffff700 1 -- 172.21.15.123:0/2805817708 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7340009e80 con 0x7f734c067660 2023-10-06T04:32:21.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f733ffff700 1 -- 172.21.15.123:0/2805817708 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734000b860 con 0x7f734c067660 2023-10-06T04:32:21.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.358+0000 7f733ffff700 1 -- 172.21.15.123:0/2805817708 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734000bbe0 con 0x7f734c067660 2023-10-06T04:32:21.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 -- 172.21.15.123:0/2805817708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c067660 msgr2=0x7f734c067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 --2- 172.21.15.123:0/2805817708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c067660 0x7f734c067a30 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f73400098e0 tx=0x7f73400091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.363 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 -- 172.21.15.123:0/2805817708 shutdown_connections 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 --2- 172.21.15.123:0/2805817708 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c067660 0x7f734c067a30 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 --2- 172.21.15.123:0/2805817708 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c0c9570 0x7f734c06e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 -- 172.21.15.123:0/2805817708 >> 172.21.15.123:0/2805817708 conn(0x7f734c0cda80 msgr2=0x7f734c0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 -- 172.21.15.123:0/2805817708 shutdown_connections 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.359+0000 7f7353a51700 1 -- 172.21.15.123:0/2805817708 wait complete. 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.360+0000 7f7353a51700 1 Processor -- start 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.360+0000 7f7353a51700 1 -- start start 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7353a51700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c067660 0x7f734c0b1b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7353a51700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734c0c9570 0x7f734c0afda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7353a51700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c0ca5b0 0x7f734c0ac8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7353a51700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0ae750 con 0x7f734c0ca5b0 2023-10-06T04:32:21.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7353a51700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0ad840 con 0x7f734c0c9570 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7353a51700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f734c0ad280 con 0x7f734c067660 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c0ca5b0 0x7f734c0ac8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c0ca5b0 0x7f734c0ac8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35650/0 (socket says 172.21.15.123:35650) 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 -- 172.21.15.123:0/523843920 learned_addr learned my addr 172.21.15.123:0/523843920 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f73517ed700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c067660 0x7f734c0b1b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7350fec700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734c0c9570 0x7f734c0afda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 -- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c067660 msgr2=0x7f734c0b1b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c067660 0x7f734c0b1b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 -- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734c0c9570 msgr2=0x7f734c0afda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734c0c9570 0x7f734c0afda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 -- 172.21.15.123:0/523843920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7340009420 con 0x7f734c0ca5b0 2023-10-06T04:32:21.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f73517ed700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c067660 0x7f734c0b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:21.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7351fee700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c0ca5b0 0x7f734c0ac8c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f734400cc10 tx=0x7f7344001970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f733d7fa700 1 -- 172.21.15.123:0/523843920 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400a930 con 0x7f734c0ca5b0 2023-10-06T04:32:21.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f733d7fa700 1 -- 172.21.15.123:0/523843920 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f734400aab0 con 0x7f734c0ca5b0 2023-10-06T04:32:21.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f734c0ad560 con 0x7f734c0ca5b0 2023-10-06T04:32:21.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.361+0000 7f733d7fa700 1 -- 172.21.15.123:0/523843920 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f734400ae70 con 0x7f734c0ca5b0 2023-10-06T04:32:21.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.362+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f734c0b2040 con 0x7f734c0ca5b0 2023-10-06T04:32:21.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.362+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f734c0ae900 con 0x7f734c0ca5b0 2023-10-06T04:32:21.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.363+0000 7f733d7fa700 1 -- 172.21.15.123:0/523843920 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7344007300 con 0x7f734c0ca5b0 2023-10-06T04:32:21.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.363+0000 7f733d7fa700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.363+0000 7f733d7fa700 1 -- 172.21.15.123:0/523843920 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f7344084760 con 0x7f734c0ca5b0 2023-10-06T04:32:21.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.363+0000 7f73517ed700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.364+0000 7f73517ed700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7340003080 tx=0x7f7340017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.367+0000 7f733d7fa700 1 -- 172.21.15.123:0/523843920 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f734c0ae900 con 0x7f734c0ca5b0 2023-10-06T04:32:21.555 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.554+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f734c0cde80 con 0x7f734c0ca5b0 2023-10-06T04:32:21.556 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.554+0000 7f733d7fa700 1 -- 172.21.15.123:0/523843920 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v30) v1 ==== 84+0+126 (secure 0 0 0) 0x7f7344051bb0 con 0x7f734c0ca5b0 2023-10-06T04:32:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:21.556 INFO:teuthology.orchestra.run.smithi123.stdout:[ 2023-10-06T04:32:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 1, 2023-10-06T04:32:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": ".mgr" 2023-10-06T04:32:21.556 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: { 2023-10-06T04:32:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "poolnum": 2, 2023-10-06T04:32:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: "poolname": "rbd" 2023-10-06T04:32:21.557 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:21.557 INFO:teuthology.orchestra.run.smithi123.stdout:] 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7330066f30 msgr2=0x7f73300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7330066f30 0x7f73300693e0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f7340003080 tx=0x7f7340017000 comp rx=0 tx=0).stop 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c0ca5b0 msgr2=0x7f734c0ac8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c0ca5b0 0x7f734c0ac8c0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f734400cc10 tx=0x7f7344001970 comp rx=0 tx=0).stop 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 shutdown_connections 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f7330066f30 0x7f73300693e0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f734c0ca5b0 0x7f734c0ac8c0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f734c067660 0x7f734c0b1b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 --2- 172.21.15.123:0/523843920 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f734c0c9570 0x7f734c0afda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 >> 172.21.15.123:0/523843920 conn(0x7f734c0cda80 msgr2=0x7f734c0a9db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:21.558 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 shutdown_connections 2023-10-06T04:32:21.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.556+0000 7f7353a51700 1 -- 172.21.15.123:0/523843920 wait complete. 2023-10-06T04:32:21.569 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-10-06T04:32:21.570 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-10-06T04:32:21.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.689+0000 7f0972cbc700 1 Processor -- start 2023-10-06T04:32:21.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.689+0000 7f0972cbc700 1 -- start start 2023-10-06T04:32:21.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0972cbc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f096c0678b0 0x7f096c0aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0972cbc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c067c80 0x7f096c0af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0972cbc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f096c06add0 con 0x7f096c158e30 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0972cbc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f096c0af710 con 0x7f096c0678b0 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0972cbc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c06c270 con 0x7f096c067c80 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0970a58700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f096c0678b0 0x7f096c0aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0971259700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f096c158e30 0x7f096c1592b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40324/0 (socket says 172.21.15.123:40324) 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.690+0000 7f0971259700 1 -- 172.21.15.123:0/2425359750 learned_addr learned my addr 172.21.15.123:0/2425359750 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f096bfff700 1 --2- 172.21.15.123:0/2425359750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c067c80 0x7f096c0af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f0970a58700 1 -- 172.21.15.123:0/2425359750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c067c80 msgr2=0x7f096c0af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f0970a58700 1 --2- 172.21.15.123:0/2425359750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c067c80 0x7f096c0af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f0970a58700 1 -- 172.21.15.123:0/2425359750 >> v1:172.21.15.123:6789/0 conn(0x7f096c158e30 legacy=0x7f096c1592b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f0970a58700 1 -- 172.21.15.123:0/2425359750 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f096c068570 con 0x7f096c0678b0 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f096b7fe700 1 -- 172.21.15.123:0/2425359750 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 586459052 0 0) 0x7f096c06add0 con 0x7f096c158e30 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f0970a58700 1 --2- 172.21.15.123:0/2425359750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c0678b0 0x7f096c0aec90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f09600098e0 tx=0x7f09600091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c38a5faa54f6fe34 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f096b7fe700 1 -- 172.21.15.123:0/2425359750 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0960009e80 con 0x7f096c0678b0 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f096b7fe700 1 -- 172.21.15.123:0/2425359750 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f096000b860 con 0x7f096c0678b0 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.691+0000 7f096b7fe700 1 -- 172.21.15.123:0/2425359750 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f096000bbe0 con 0x7f096c0678b0 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 -- 172.21.15.123:0/2425359750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c0678b0 msgr2=0x7f096c0aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 --2- 172.21.15.123:0/2425359750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c0678b0 0x7f096c0aec90 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f09600098e0 tx=0x7f09600091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 -- 172.21.15.123:0/2425359750 shutdown_connections 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 --2- 172.21.15.123:0/2425359750 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c0678b0 0x7f096c0aec90 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 --2- 172.21.15.123:0/2425359750 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c067c80 0x7f096c0af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 -- 172.21.15.123:0/2425359750 >> 172.21.15.123:0/2425359750 conn(0x7f096c0cdb10 msgr2=0x7f096c0abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:21.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 -- 172.21.15.123:0/2425359750 shutdown_connections 2023-10-06T04:32:21.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.692+0000 7f0972cbc700 1 -- 172.21.15.123:0/2425359750 wait complete. 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.693+0000 7f0972cbc700 1 Processor -- start 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.693+0000 7f0972cbc700 1 -- start start 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0972cbc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c0678b0 0x7f096c15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0972cbc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c067c80 0x7f096c1435b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0972cbc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c158e30 0x7f096c143af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0972cbc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c144030 con 0x7f096c067c80 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0972cbc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f096c15c780 con 0x7f096c158e30 2023-10-06T04:32:21.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0972cbc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f096c15ca60 con 0x7f096c0678b0 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f096bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c067c80 0x7f096c1435b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f096bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c067c80 0x7f096c1435b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35672/0 (socket says 172.21.15.123:35672) 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f096bfff700 1 -- 172.21.15.123:0/1630242307 learned_addr learned my addr 172.21.15.123:0/1630242307 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0970a58700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c0678b0 0x7f096c15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.694+0000 7f0971259700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c158e30 0x7f096c143af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f096bfff700 1 -- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c0678b0 msgr2=0x7f096c15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f096bfff700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c0678b0 0x7f096c15f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f096bfff700 1 -- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c158e30 msgr2=0x7f096c143af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f096bfff700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c158e30 0x7f096c143af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:21.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f096bfff700 1 -- 172.21.15.123:0/1630242307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0960009420 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f096bfff700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c067c80 0x7f096c1435b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f095800d890 tx=0x7f0958008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f0968ff9700 1 -- 172.21.15.123:0/1630242307 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800b600 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f0968ff9700 1 -- 172.21.15.123:0/1630242307 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f095800b760 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f0968ff9700 1 -- 172.21.15.123:0/1630242307 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f095800bae0 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f096c15cd40 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.695+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f096c15d300 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.696+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f096c0ac2d0 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.696+0000 7f0968ff9700 1 -- 172.21.15.123:0/1630242307 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f095801e460 con 0x7f096c067c80 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.697+0000 7f0968ff9700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0950066f30 0x7f09500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:21.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.697+0000 7f0968ff9700 1 -- 172.21.15.123:0/1630242307 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 5206+0+0 (secure 0 0 0) 0x7f0958084a20 con 0x7f096c067c80 2023-10-06T04:32:21.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.697+0000 7f0970a58700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0950066f30 0x7f09500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:21.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.697+0000 7f0970a58700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0950066f30 0x7f09500693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f09600091e0 tx=0x7f0960017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:21.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.701+0000 7f0968ff9700 1 -- 172.21.15.123:0/1630242307 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0958051e80 con 0x7f096c067c80 2023-10-06T04:32:21.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:21.883+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f096c0af930 con 0x7f096c067c80 2023-10-06T04:32:22.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.424+0000 7f0968ff9700 1 -- 172.21.15.123:0/1630242307 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v31) v1 ==== 138+0+0 (secure 0 0 0) 0x7f09580554a0 con 0x7f096c067c80 2023-10-06T04:32:22.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.427+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0950066f30 msgr2=0x7f09500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:22.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.427+0000 7f0972cbc700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0950066f30 0x7f09500693e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f09600091e0 tx=0x7f0960017000 comp rx=0 tx=0).stop 2023-10-06T04:32:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.427+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c067c80 msgr2=0x7f096c1435b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.427+0000 7f0972cbc700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c067c80 0x7f096c1435b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f095800d890 tx=0x7f0958008040 comp rx=0 tx=0).stop 2023-10-06T04:32:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 shutdown_connections 2023-10-06T04:32:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0950066f30 0x7f09500693e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f096c158e30 0x7f096c143af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f096c0678b0 0x7f096c15f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 --2- 172.21.15.123:0/1630242307 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f096c067c80 0x7f096c1435b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 >> 172.21.15.123:0/1630242307 conn(0x7f096c0cdb10 msgr2=0x7f096c158350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 shutdown_connections 2023-10-06T04:32:22.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.428+0000 7f0972cbc700 1 -- 172.21.15.123:0/1630242307 wait complete. 2023-10-06T04:32:22.431 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' created 2023-10-06T04:32:22.446 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:32:22.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.571+0000 7f608561b700 1 Processor -- start 2023-10-06T04:32:22.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.571+0000 7f608561b700 1 -- start start 2023-10-06T04:32:22.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.572+0000 7f608561b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 0x7f6080161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.572+0000 7f608561b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f608006a4e0 con 0x7f60800c8a10 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.572+0000 7f608561b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f60800c8ec0 con 0x7f608013fa70 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.572+0000 7f608561b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f60800c91a0 con 0x7f608006e9f0 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.572+0000 7f607e7fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f60800c8a10 0x7f608006e2e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40338/0 (socket says 172.21.15.123:40338) 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.572+0000 7f607e7fc700 1 -- 172.21.15.123:0/1039340627 learned_addr learned my addr 172.21.15.123:0/1039340627 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607f7fe700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 0x7f6080161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4006715383 0 0) 0x7f608006a4e0 con 0x7f60800c8a10 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6068003570 con 0x7f60800c8a10 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579571612 0 0) 0x7f60800c8ec0 con 0x7f608013fa70 2023-10-06T04:32:22.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f608006a4e0 con 0x7f608013fa70 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1725273816 0 0) 0x7f6068003570 con 0x7f60800c8a10 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 msgr2=0x7f6080161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 0x7f6080161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 >> v1:172.21.15.177:6789/0 conn(0x7f608013fa70 legacy=0x7f60800c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60800c9480 con 0x7f60800c8a10 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f6070001470 con 0x7f60800c8a10 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 >> v1:172.21.15.123:6789/0 conn(0x7f60800c8a10 legacy=0x7f608006e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6068003c20 0x7f6068004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6068004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 0x7f6068005220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800c91a0 con 0x7f6068004810 2023-10-06T04:32:22.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f606800bbf0 con 0x7f6068003c20 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f606800bdb0 con 0x7f608006e9f0 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6068004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607f7fe700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 0x7f6068005220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607effd700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6068003c20 0x7f6068004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607e7fc700 1 -- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 msgr2=0x7f6068005220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f608006e9f0 0x7f6068005220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607e7fc700 1 -- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6068003c20 msgr2=0x7f6068004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6068003c20 0x7f6068004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607e7fc700 1 -- 172.21.15.123:0/1039340627 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6068003570 con 0x7f6068004810 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.573+0000 7f607effd700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6068003c20 0x7f6068004010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6068004c80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f607000c1e0 tx=0x7f6070001d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff907a844e791015 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:22.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f607000cb10 con 0x7f6068004810 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f607000cc90 con 0x7f6068004810 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f607dffb700 1 -- 172.21.15.123:0/1039340627 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6070005b70 con 0x7f6068004810 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f608561b700 1 -- 172.21.15.123:0/1039340627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 msgr2=0x7f6068004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f608561b700 1 --2- 172.21.15.123:0/1039340627 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6068004c80 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f607000c1e0 tx=0x7f6070001d40 comp rx=0 tx=0).stop 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f607f7fe700 1 -- 172.21.15.123:0/1039340627 reap_dead start 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f608561b700 1 -- 172.21.15.123:0/1039340627 shutdown_connections 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f608561b700 1 -- 172.21.15.123:0/1039340627 >> 172.21.15.123:0/1039340627 conn(0x7f60800cd020 msgr2=0x7f60800cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f608561b700 1 -- 172.21.15.123:0/1039340627 shutdown_connections 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.574+0000 7f608561b700 1 -- 172.21.15.123:0/1039340627 wait complete. 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.575+0000 7f608561b700 1 Processor -- start 2023-10-06T04:32:22.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f608561b700 1 -- start start 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f608561b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6080155970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f608561b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60800aee30 0x7f60800cbda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f608561b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60800af2a0 0x7f60800cc2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f608561b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800c6230 con 0x7f6068004810 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f608561b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f60800ace20 con 0x7f60800aee30 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f608561b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6080155eb0 con 0x7f60800af2a0 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f607effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6080155970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f607effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6080155970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35690/0 (socket says 172.21.15.123:35690) 2023-10-06T04:32:22.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f607effd700 1 -- 172.21.15.123:0/1672180308 learned_addr learned my addr 172.21.15.123:0/1672180308 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60800aee30 0x7f60800cbda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.576+0000 7f607f7fe700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60800af2a0 0x7f60800cc2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f607effd700 1 -- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60800af2a0 msgr2=0x7f60800cc2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f607effd700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60800af2a0 0x7f60800cc2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f607effd700 1 -- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60800aee30 msgr2=0x7f60800cbda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f607effd700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60800aee30 0x7f60800cbda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f607effd700 1 -- 172.21.15.123:0/1672180308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f60800c6420 con 0x7f6068004810 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60800aee30 0x7f60800cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f607effd700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6080155970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f607400e830 tx=0x7f6074009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f6066ffd700 1 -- 172.21.15.123:0/1672180308 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074006820 con 0x7f6068004810 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f6066ffd700 1 -- 172.21.15.123:0/1672180308 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6074006980 con 0x7f6068004810 2023-10-06T04:32:22.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f60800c6700 con 0x7f6068004810 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f6066ffd700 1 -- 172.21.15.123:0/1672180308 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6074006d20 con 0x7f6068004810 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.577+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f60800ad070 con 0x7f6068004810 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.578+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f60800ad290 con 0x7f6068004810 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.578+0000 7f6066ffd700 1 -- 172.21.15.123:0/1672180308 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6074007230 con 0x7f6068004810 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.579+0000 7f6066ffd700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f606c066f30 0x7f606c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.579+0000 7f6066ffd700 1 -- 172.21.15.123:0/1672180308 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 5576+0+0 (secure 0 0 0) 0x7f6074084800 con 0x7f6068004810 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.579+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f606c066f30 0x7f606c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:22.585 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.579+0000 7f607e7fc700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f606c066f30 0x7f606c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f60800ccb20 tx=0x7f6070015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:22.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.583+0000 7f6066ffd700 1 -- 172.21.15.123:0/1672180308 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6074051af0 con 0x7f6068004810 2023-10-06T04:32:22.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:22.764+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f60800c3cd0 con 0x7f6068004810 2023-10-06T04:32:23.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.431+0000 7f6066ffd700 1 -- 172.21.15.123:0/1672180308 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v32) v1 ==== 170+0+0 (secure 0 0 0) 0x7f60740170d0 con 0x7f6068004810 2023-10-06T04:32:23.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.436+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f606c066f30 msgr2=0x7f606c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:23.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.436+0000 7f608561b700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f606c066f30 0x7f606c0693e0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f60800ccb20 tx=0x7f6070015000 comp rx=0 tx=0).stop 2023-10-06T04:32:23.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.436+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 msgr2=0x7f6080155970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:23.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.436+0000 7f608561b700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6080155970 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f607400e830 tx=0x7f6074009040 comp rx=0 tx=0).stop 2023-10-06T04:32:23.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.443+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 shutdown_connections 2023-10-06T04:32:23.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.443+0000 7f608561b700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f606c066f30 0x7f606c0693e0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.443+0000 7f608561b700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6068004810 0x7f6080155970 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.443+0000 7f608561b700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f60800af2a0 0x7f60800cc2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.443+0000 7f608561b700 1 --2- 172.21.15.123:0/1672180308 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f60800aee30 0x7f60800cbda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.443+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 >> 172.21.15.123:0/1672180308 conn(0x7f60800cd020 msgr2=0x7f608015db50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:23.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.444+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 shutdown_connections 2023-10-06T04:32:23.446 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.444+0000 7f608561b700 1 -- 172.21.15.123:0/1672180308 wait complete. 2023-10-06T04:32:23.449 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' created 2023-10-06T04:32:23.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-10-06T04:32:23.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.586+0000 7fa86ccb0700 1 Processor -- start 2023-10-06T04:32:23.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.586+0000 7fa86ccb0700 1 -- start start 2023-10-06T04:32:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86ccb0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa8680c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86ccb0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa868068670 0x7fa868068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:23.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86ccb0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa868069aa0 con 0x7fa8680c8340 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86ccb0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8680aa500 con 0x7fa868068670 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86ccb0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa868069050 con 0x7fa8680c8710 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa866d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa868068670 0x7fa868068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86659c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa8680c8340 0x7fa8680a9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40342/0 (socket says 172.21.15.123:40342) 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86659c700 1 -- 172.21.15.123:0/2770477946 learned_addr learned my addr 172.21.15.123:0/2770477946 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa865d9b700 1 --2- 172.21.15.123:0/2770477946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa8680c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa866d9d700 1 -- 172.21.15.123:0/2770477946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 msgr2=0x7fa8680c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa866d9d700 1 --2- 172.21.15.123:0/2770477946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa8680c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa866d9d700 1 -- 172.21.15.123:0/2770477946 >> v1:172.21.15.123:6789/0 conn(0x7fa8680c8340 legacy=0x7fa8680a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa866d9d700 1 -- 172.21.15.123:0/2770477946 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa86806a340 con 0x7fa868068670 2023-10-06T04:32:23.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa866d9d700 1 --2- 172.21.15.123:0/2770477946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa868068b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa860009ec0 tx=0x7fa860009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd245bbcc7ad26b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86559a700 1 -- 172.21.15.123:0/2770477946 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa86000e070 con 0x7fa868068670 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86559a700 1 -- 172.21.15.123:0/2770477946 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa86000bbb0 con 0x7fa868068670 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.587+0000 7fa86559a700 1 -- 172.21.15.123:0/2770477946 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa860012430 con 0x7fa868068670 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 -- 172.21.15.123:0/2770477946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 msgr2=0x7fa868068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/2770477946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa868068b10 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7fa860009ec0 tx=0x7fa860009780 comp rx=0 tx=0).stop 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 -- 172.21.15.123:0/2770477946 shutdown_connections 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/2770477946 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa868068b10 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/2770477946 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa8680c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 -- 172.21.15.123:0/2770477946 >> 172.21.15.123:0/2770477946 conn(0x7fa8680c9ca0 msgr2=0x7fa86806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 -- 172.21.15.123:0/2770477946 shutdown_connections 2023-10-06T04:32:23.591 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.588+0000 7fa86ccb0700 1 -- 172.21.15.123:0/2770477946 wait complete. 2023-10-06T04:32:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.589+0000 7fa86ccb0700 1 Processor -- start 2023-10-06T04:32:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.589+0000 7fa86ccb0700 1 -- start start 2023-10-06T04:32:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86ccb0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa8680a9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86ccb0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8680c8340 0x7fa86815b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86ccb0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa868169930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:23.594 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86ccb0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680c7880 con 0x7fa868068670 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86ccb0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8680aa270 con 0x7fa8680c8340 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86ccb0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa86816ae20 con 0x7fa8680c8710 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa8680a9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa8680a9d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35710/0 (socket says 172.21.15.123:35710) 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 -- 172.21.15.123:0/4293125998 learned_addr learned my addr 172.21.15.123:0/4293125998 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa866d9d700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa868169930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa865d9b700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8680c8340 0x7fa86815b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 -- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 msgr2=0x7fa868169930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa868169930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 -- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8680c8340 msgr2=0x7fa86815b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8680c8340 0x7fa86815b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 -- 172.21.15.123:0/4293125998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa860009a20 con 0x7fa868068670 2023-10-06T04:32:23.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa865d9b700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8680c8340 0x7fa86815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa86659c700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa8680a9d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa85800c7c0 tx=0x7fa858007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.590+0000 7fa8567fc700 1 -- 172.21.15.123:0/4293125998 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858005b30 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.591+0000 7fa8567fc700 1 -- 172.21.15.123:0/4293125998 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa858005cb0 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.591+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa86816ab20 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.591+0000 7fa8567fc700 1 -- 172.21.15.123:0/4293125998 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa858004a80 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.591+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa86815bdd0 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.592+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8680aa6f0 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.592+0000 7fa8567fc700 1 -- 172.21.15.123:0/4293125998 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8580070d0 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.592+0000 7fa8567fc700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa848066f30 0x7fa8480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.592+0000 7fa8567fc700 1 -- 172.21.15.123:0/4293125998 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5958+0+0 (secure 0 0 0) 0x7fa8580838c0 con 0x7fa868068670 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.592+0000 7fa865d9b700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa848066f30 0x7fa8480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:23.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.593+0000 7fa865d9b700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa848066f30 0x7fa8480693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa85c006fd0 tx=0x7fa85c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:23.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.597+0000 7fa8567fc700 1 -- 172.21.15.123:0/4293125998 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa85804b5a0 con 0x7fa868068670 2023-10-06T04:32:23.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:23.821+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa8680ab670 con 0x7fa868068670 2023-10-06T04:32:24.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.430+0000 7fa8567fc700 1 -- 172.21.15.123:0/4293125998 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 5 and data pool 6 v13) v1 ==== 172+0+0 (secure 0 0 0) 0x7fa858050a30 con 0x7fa868068670 2023-10-06T04:32:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa848066f30 msgr2=0x7fa8480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa848066f30 0x7fa8480693e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fa85c006fd0 tx=0x7fa85c008040 comp rx=0 tx=0).stop 2023-10-06T04:32:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 msgr2=0x7fa8680a9d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa8680a9d30 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fa85800c7c0 tx=0x7fa858007f70 comp rx=0 tx=0).stop 2023-10-06T04:32:24.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 shutdown_connections 2023-10-06T04:32:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa848066f30 0x7fa8480693e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa868068670 0x7fa8680a9d30 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8680c8710 0x7fa868169930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 --2- 172.21.15.123:0/4293125998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa8680c8340 0x7fa86815b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 >> 172.21.15.123:0/4293125998 conn(0x7fa8680c9ca0 msgr2=0x7fa868158980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.433+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 shutdown_connections 2023-10-06T04:32:24.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.434+0000 7fa86ccb0700 1 -- 172.21.15.123:0/4293125998 wait complete. 2023-10-06T04:32:24.436 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 5 and data pool 6 2023-10-06T04:32:24.450 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:24.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.567+0000 7fd70e80e700 1 Processor -- start 2023-10-06T04:32:24.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.567+0000 7fd70e80e700 1 -- start start 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd70e80e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd7080688c0 0x7fd70806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd70e80e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7080cc190 con 0x7fd708068c90 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd70e80e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7080cb580 con 0x7fd70806a930 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd70e80e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7080cc620 con 0x7fd7080688c0 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd707fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd7080688c0 0x7fd70806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd70cdab700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd708068c90 0x7fd7080c3840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40348/0 (socket says 172.21.15.123:40348) 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd70cdab700 1 -- 172.21.15.123:0/1519278441 learned_addr learned my addr 172.21.15.123:0/1519278441 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd707fff700 1 -- 172.21.15.123:0/1519278441 >> v1:172.21.15.177:6789/0 conn(0x7fd70806a930 legacy=0x7fd708156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd707fff700 1 -- 172.21.15.123:0/1519278441 >> v1:172.21.15.123:6789/0 conn(0x7fd708068c90 legacy=0x7fd7080c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd707fff700 1 -- 172.21.15.123:0/1519278441 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7080cb860 con 0x7fd7080688c0 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd707fff700 1 --2- 172.21.15.123:0/1519278441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd70806a3f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0098e0 tx=0x7fd6fc0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99fc902640c7c820 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd706ffd700 1 -- 172.21.15.123:0/1519278441 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc009e80 con 0x7fd7080688c0 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd706ffd700 1 -- 172.21.15.123:0/1519278441 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc00b820 con 0x7fd7080688c0 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.568+0000 7fd706ffd700 1 -- 172.21.15.123:0/1519278441 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00bba0 con 0x7fd7080688c0 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.569+0000 7fd70e80e700 1 -- 172.21.15.123:0/1519278441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 msgr2=0x7fd70806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.569+0000 7fd70e80e700 1 --2- 172.21.15.123:0/1519278441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd70806a3f0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0098e0 tx=0x7fd6fc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.569+0000 7fd70e80e700 1 -- 172.21.15.123:0/1519278441 shutdown_connections 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.569+0000 7fd70e80e700 1 --2- 172.21.15.123:0/1519278441 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd70806a3f0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.569+0000 7fd70e80e700 1 -- 172.21.15.123:0/1519278441 >> 172.21.15.123:0/1519278441 conn(0x7fd7080c9c90 msgr2=0x7fd70806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.569+0000 7fd70e80e700 1 -- 172.21.15.123:0/1519278441 shutdown_connections 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.569+0000 7fd70e80e700 1 -- 172.21.15.123:0/1519278441 wait complete. 2023-10-06T04:32:24.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.570+0000 7fd70e80e700 1 Processor -- start 2023-10-06T04:32:24.574 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.570+0000 7fd70e80e700 1 -- start start 2023-10-06T04:32:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd70e80e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd7080af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd70e80e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd708068c90 0x7fd7080c1b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd70e80e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd70806a930 0x7fd70816cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd70e80e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7080b1980 con 0x7fd7080688c0 2023-10-06T04:32:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd70e80e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd708155b50 con 0x7fd70806a930 2023-10-06T04:32:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd70e80e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd70816de30 con 0x7fd708068c90 2023-10-06T04:32:24.576 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd707fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd7080af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd707fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd7080af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35724/0 (socket says 172.21.15.123:35724) 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd707fff700 1 -- 172.21.15.123:0/820641231 learned_addr learned my addr 172.21.15.123:0/820641231 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd70cdab700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd70806a930 0x7fd70816cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.571+0000 7fd7077fe700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd708068c90 0x7fd7080c1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd707fff700 1 -- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd708068c90 msgr2=0x7fd7080c1b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd707fff700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd708068c90 0x7fd7080c1b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd707fff700 1 -- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd70806a930 msgr2=0x7fd70816cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd707fff700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd70806a930 0x7fd70816cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd707fff700 1 -- 172.21.15.123:0/820641231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6fc009420 con 0x7fd7080688c0 2023-10-06T04:32:24.577 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd707fff700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd7080af6a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0030c0 tx=0x7fd6fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd6effff700 1 -- 172.21.15.123:0/820641231 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00ba60 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd6effff700 1 -- 172.21.15.123:0/820641231 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6fc020070 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd6effff700 1 -- 172.21.15.123:0/820641231 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6fc00bbe0 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd70816db50 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.572+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7081590d0 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.573+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd70806cc70 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.573+0000 7fd6effff700 1 -- 172.21.15.123:0/820641231 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd6fc022030 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.573+0000 7fd6effff700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd6f4066f30 0x7fd6f40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.574+0000 7fd6effff700 1 -- 172.21.15.123:0/820641231 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fd6fc086540 con 0x7fd7080688c0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.574+0000 7fd7077fe700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd6f4066f30 0x7fd6f40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.578 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.574+0000 7fd7077fe700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd6f4066f30 0x7fd6f40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7080ae6f0 tx=0x7fd6f8009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:24.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.578+0000 7fd6effff700 1 -- 172.21.15.123:0/820641231 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd6fc053650 con 0x7fd7080688c0 2023-10-06T04:32:24.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.759+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd70804bb80 con 0x7fd7080688c0 2023-10-06T04:32:24.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.760+0000 7fd6effff700 1 -- 172.21.15.123:0/820641231 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+17023 (secure 0 0 0) 0x7fd6fc056c70 con 0x7fd7080688c0 2023-10-06T04:32:24.762 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:24.762 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":33,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:23.822405+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:24.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd6f4066f30 msgr2=0x7fd6f40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd6f4066f30 0x7fd6f40693e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7fd7080ae6f0 tx=0x7fd6f8009060 comp rx=0 tx=0).stop 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 msgr2=0x7fd7080af6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd7080af6a0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fd6fc0030c0 tx=0x7fd6fc016000 comp rx=0 tx=0).stop 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 shutdown_connections 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd6f4066f30 0x7fd6f40693e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd7080688c0 0x7fd7080af6a0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd70806a930 0x7fd70816cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 --2- 172.21.15.123:0/820641231 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd708068c90 0x7fd7080c1b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 >> 172.21.15.123:0/820641231 conn(0x7fd7080c9c90 msgr2=0x7fd708155ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:24.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 shutdown_connections 2023-10-06T04:32:24.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.762+0000 7fd70e80e700 1 -- 172.21.15.123:0/820641231 wait complete. 2023-10-06T04:32:24.776 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:24.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.891+0000 7ff79564e700 1 Processor -- start 2023-10-06T04:32:24.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.891+0000 7ff79564e700 1 -- start start 2023-10-06T04:32:24.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.892+0000 7ff79564e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 0x7ff790067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.892+0000 7ff79564e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff7900aa950 0x7ff79006e960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.892+0000 7ff79564e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff790067f70 con 0x7ff7900c9570 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.892+0000 7ff79564e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79006a750 con 0x7ff7900aa950 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.892+0000 7ff79564e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79006aa30 con 0x7ff790067660 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.892+0000 7ff78e7fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff7900c9570 0x7ff79006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40356/0 (socket says 172.21.15.123:40356) 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78e7fc700 1 -- 172.21.15.123:0/371998083 learned_addr learned my addr 172.21.15.123:0/371998083 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78effd700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 0x7ff790067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78f7fe700 1 --2- 172.21.15.123:0/371998083 >> v2:172.21.15.177:3300/0 conn(0x7ff7900aa950 0x7ff79006e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2660913995 0 0) 0x7ff790067f70 con 0x7ff7900c9570 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff778003570 con 0x7ff7900c9570 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2029710474 0 0) 0x7ff778003570 con 0x7ff7900c9570 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 msgr2=0x7ff790067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:24.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 0x7ff790067a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 >> v2:172.21.15.177:3300/0 conn(0x7ff7900aa950 msgr2=0x7ff79006e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 --2- 172.21.15.123:0/371998083 >> v2:172.21.15.177:3300/0 conn(0x7ff7900aa950 0x7ff79006e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79006ad10 con 0x7ff7900c9570 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7ff784001470 con 0x7ff7900c9570 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 >> v1:172.21.15.123:6789/0 conn(0x7ff7900c9570 legacy=0x7ff79006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff778003a60 0x7ff778003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 0x7ff778004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff778004d50 0x7ff7780051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79006a750 con 0x7ff778004d50 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79006aa30 con 0x7ff778003a60 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff77800ba60 con 0x7ff790067660 2023-10-06T04:32:24.904 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78f7fe700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff778004d50 0x7ff7780051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78effd700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff778003a60 0x7ff778003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78e7fc700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 0x7ff778004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78f7fe700 1 -- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 msgr2=0x7ff778004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78f7fe700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff790067660 0x7ff778004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78f7fe700 1 -- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff778003a60 msgr2=0x7ff778003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78f7fe700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff778003a60 0x7ff778003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.893+0000 7ff78f7fe700 1 -- 172.21.15.123:0/371998083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff778003570 con 0x7ff778004d50 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff78f7fe700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff778004d50 0x7ff7780051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e210 tx=0x7ff78800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60b0aebc5c7e61c4 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800ea60 con 0x7ff778004d50 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78800f040 con 0x7ff778004d50 2023-10-06T04:32:24.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff78dffb700 1 -- 172.21.15.123:0/371998083 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78800ee10 con 0x7ff778004d50 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff79564e700 1 -- 172.21.15.123:0/371998083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff778004d50 msgr2=0x7ff7780051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff79564e700 1 --2- 172.21.15.123:0/371998083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff778004d50 0x7ff7780051c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ff78800e210 tx=0x7ff78800b040 comp rx=0 tx=0).stop 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff78f7fe700 1 -- 172.21.15.123:0/371998083 reap_dead start 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff79564e700 1 -- 172.21.15.123:0/371998083 shutdown_connections 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff79564e700 1 -- 172.21.15.123:0/371998083 >> 172.21.15.123:0/371998083 conn(0x7ff7900cda80 msgr2=0x7ff7900c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff79564e700 1 -- 172.21.15.123:0/371998083 shutdown_connections 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.894+0000 7ff79564e700 1 -- 172.21.15.123:0/371998083 wait complete. 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 Processor -- start 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 -- start start 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7900adb50 0x7ff790155b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7900adf20 0x7ff7901598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff79014d600 0x7ff79014daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79015ace0 con 0x7ff7900adf20 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff790159e30 con 0x7ff7900adb50 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.896+0000 7ff79564e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79015a0d0 con 0x7ff79014d600 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7900adf20 0x7ff7901598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7900adf20 0x7ff7901598f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35746/0 (socket says 172.21.15.123:35746) 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 -- 172.21.15.123:0/1204097980 learned_addr learned my addr 172.21.15.123:0/1204097980 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78f7fe700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff79014d600 0x7ff79014daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78effd700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7900adb50 0x7ff790155b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 -- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff79014d600 msgr2=0x7ff79014daa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff79014d600 0x7ff79014daa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 -- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7900adb50 msgr2=0x7ff790155b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:24.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7900adb50 0x7ff790155b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:24.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 -- 172.21.15.123:0/1204097980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79015a3b0 con 0x7ff7900adf20 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff78e7fc700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7900adf20 0x7ff7901598f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff784000fd0 tx=0x7ff784000cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff776ffd700 1 -- 172.21.15.123:0/1204097980 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400a520 con 0x7ff7900adf20 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff776ffd700 1 -- 172.21.15.123:0/1204097980 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff784010040 con 0x7ff7900adf20 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79015a690 con 0x7ff7900adf20 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff776ffd700 1 -- 172.21.15.123:0/1204097980 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400a8d0 con 0x7ff7900adf20 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.897+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79014dfe0 con 0x7ff7900adf20 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.898+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79015ae50 con 0x7ff7900adf20 2023-10-06T04:32:24.909 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.898+0000 7ff776ffd700 1 -- 172.21.15.123:0/1204097980 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff78401d3b0 con 0x7ff7900adf20 2023-10-06T04:32:24.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.899+0000 7ff776ffd700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff77c066f30 0x7ff77c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:24.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.899+0000 7ff776ffd700 1 -- 172.21.15.123:0/1204097980 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff784083d70 con 0x7ff7900adf20 2023-10-06T04:32:24.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.899+0000 7ff78effd700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff77c066f30 0x7ff77c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:24.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.899+0000 7ff78effd700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff77c066f30 0x7ff77c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff780006f00 tx=0x7ff7800091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:24.910 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:24.903+0000 7ff776ffd700 1 -- 172.21.15.123:0/1204097980 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff784050e80 con 0x7ff7900adf20 2023-10-06T04:32:25.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.086+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff7900cde80 con 0x7ff7900adf20 2023-10-06T04:32:25.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.086+0000 7ff776ffd700 1 -- 172.21.15.123:0/1204097980 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v33) v1 ==== 74+0+17023 (secure 0 0 0) 0x7ff784019070 con 0x7ff7900adf20 2023-10-06T04:32:25.088 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:25.088 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":33,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:23.822405+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:25.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.088+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff77c066f30 msgr2=0x7ff77c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:25.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.088+0000 7ff79564e700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff77c066f30 0x7ff77c0693e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7ff780006f00 tx=0x7ff7800091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.088+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7900adf20 msgr2=0x7ff7901598f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.088+0000 7ff79564e700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7900adf20 0x7ff7901598f0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7ff784000fd0 tx=0x7ff784000cc0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 shutdown_connections 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff77c066f30 0x7ff77c0693e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff79014d600 0x7ff79014daa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff7900adb50 0x7ff790155b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 --2- 172.21.15.123:0/1204097980 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff7900adf20 0x7ff7901598f0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 >> 172.21.15.123:0/1204097980 conn(0x7ff7900cda80 msgr2=0x7ff790158860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 shutdown_connections 2023-10-06T04:32:25.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.089+0000 7ff79564e700 1 -- 172.21.15.123:0/1204097980 wait complete. 2023-10-06T04:32:25.102 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-10-06T04:32:25.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.221+0000 7fb684552700 1 Processor -- start 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.221+0000 7fb684552700 1 -- start start 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb684552700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb67c158670 0x7fb67c067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb684552700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c06a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb684552700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb67c068000 con 0x7fb67c154ee0 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb684552700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb67c0c7d70 con 0x7fb67c158670 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb684552700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c06ecb0 con 0x7fb67c067ba0 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb681aed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb67c158670 0x7fb67c067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb6822ee700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fb67c154ee0 0x7fb67c0c7660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40360/0 (socket says 172.21.15.123:40360) 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.222+0000 7fb6822ee700 1 -- 172.21.15.123:0/1799454481 learned_addr learned my addr 172.21.15.123:0/1799454481 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb682aef700 1 --2- 172.21.15.123:0/1799454481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c06a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:25.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb681aed700 1 -- 172.21.15.123:0/1799454481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 msgr2=0x7fb67c06a3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb681aed700 1 --2- 172.21.15.123:0/1799454481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c06a3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb681aed700 1 -- 172.21.15.123:0/1799454481 >> v1:172.21.15.123:6789/0 conn(0x7fb67c154ee0 legacy=0x7fb67c0c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb681aed700 1 -- 172.21.15.123:0/1799454481 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb67c0c7f90 con 0x7fb67c158670 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb6812ec700 1 -- 172.21.15.123:0/1799454481 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 757251374 0 0) 0x7fb66c001110 con 0x7fb67c154ee0 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb681aed700 1 --2- 172.21.15.123:0/1799454481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c158670 0x7fb67c067660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb6780098e0 tx=0x7fb6780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6143bb1bc654d476 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb6812ec700 1 -- 172.21.15.123:0/1799454481 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800b640 con 0x7fb67c158670 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb6812ec700 1 -- 172.21.15.123:0/1799454481 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb67800b7a0 con 0x7fb67c158670 2023-10-06T04:32:25.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.223+0000 7fb6812ec700 1 -- 172.21.15.123:0/1799454481 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800bb40 con 0x7fb67c158670 2023-10-06T04:32:25.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 -- 172.21.15.123:0/1799454481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c158670 msgr2=0x7fb67c067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:25.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 --2- 172.21.15.123:0/1799454481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c158670 0x7fb67c067660 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb6780098e0 tx=0x7fb6780091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 -- 172.21.15.123:0/1799454481 shutdown_connections 2023-10-06T04:32:25.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 --2- 172.21.15.123:0/1799454481 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c06a3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 --2- 172.21.15.123:0/1799454481 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c158670 0x7fb67c067660 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 -- 172.21.15.123:0/1799454481 >> 172.21.15.123:0/1799454481 conn(0x7fb67c0cd020 msgr2=0x7fb67c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 -- 172.21.15.123:0/1799454481 shutdown_connections 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.224+0000 7fb684552700 1 -- 172.21.15.123:0/1799454481 wait complete. 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.225+0000 7fb684552700 1 Processor -- start 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.225+0000 7fb684552700 1 -- start start 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c0cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c154ee0 0x7fb67c16a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c158670 0x7fb67c0aef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c16a930 con 0x7fb67c154ee0 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c16d8f0 con 0x7fb67c158670 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb67c0af450 con 0x7fb67c067ba0 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c154ee0 0x7fb67c16a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:25.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c154ee0 0x7fb67c16a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35764/0 (socket says 172.21.15.123:35764) 2023-10-06T04:32:25.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 -- 172.21.15.123:0/3488555693 learned_addr learned my addr 172.21.15.123:0/3488555693 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:25.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb6822ee700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c0cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:25.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb682aef700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c158670 0x7fb67c0aef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:25.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 -- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 msgr2=0x7fb67c0cb4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:25.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c0cb4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 -- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c158670 msgr2=0x7fb67c0aef10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:25.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c158670 0x7fb67c0aef10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 -- 172.21.15.123:0/3488555693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb678009420 con 0x7fb67c154ee0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb682aef700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c158670 0x7fb67c0aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb6822ee700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c0cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb681aed700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c154ee0 0x7fb67c16a3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb67c063d40 tx=0x7fb678017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb6727fc700 1 -- 172.21.15.123:0/3488555693 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb67800bb20 con 0x7fb67c154ee0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb6727fc700 1 -- 172.21.15.123:0/3488555693 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb678011420 con 0x7fb67c154ee0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb6727fc700 1 -- 172.21.15.123:0/3488555693 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6780117c0 con 0x7fb67c154ee0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb67c0cbb50 con 0x7fb67c154ee0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.226+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb67c16d2a0 con 0x7fb67c154ee0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.227+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb67c16da60 con 0x7fb67c154ee0 2023-10-06T04:32:25.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.228+0000 7fb6727fc700 1 -- 172.21.15.123:0/3488555693 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb678011b70 con 0x7fb67c154ee0 2023-10-06T04:32:25.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.228+0000 7fb6727fc700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb660066f30 0x7fb6600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:25.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.228+0000 7fb6727fc700 1 -- 172.21.15.123:0/3488555693 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fb678085da0 con 0x7fb67c154ee0 2023-10-06T04:32:25.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.228+0000 7fb6822ee700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb660066f30 0x7fb6600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:25.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.228+0000 7fb6822ee700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb660066f30 0x7fb6600693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb66c001890 tx=0x7fb66c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:25.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.232+0000 7fb6727fc700 1 -- 172.21.15.123:0/3488555693 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb678052eb0 con 0x7fb67c154ee0 2023-10-06T04:32:25.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:25.463+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fb67c04bb80 con 0x7fb67c154ee0 2023-10-06T04:32:26.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.453+0000 7fb6727fc700 1 -- 172.21.15.123:0/3488555693 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v16) v1 ==== 118+0+0 (secure 0 0 0) 0x7fb6780564d0 con 0x7fb67c154ee0 2023-10-06T04:32:26.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.455+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb660066f30 msgr2=0x7fb6600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:26.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.455+0000 7fb684552700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb660066f30 0x7fb6600693e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fb66c001890 tx=0x7fb66c009040 comp rx=0 tx=0).stop 2023-10-06T04:32:26.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.455+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c154ee0 msgr2=0x7fb67c16a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:26.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.455+0000 7fb684552700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c154ee0 0x7fb67c16a3f0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb67c063d40 tx=0x7fb678017000 comp rx=0 tx=0).stop 2023-10-06T04:32:26.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 shutdown_connections 2023-10-06T04:32:26.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb660066f30 0x7fb6600693e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb67c158670 0x7fb67c0aef10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb67c067ba0 0x7fb67c0cb4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 --2- 172.21.15.123:0/3488555693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb67c154ee0 0x7fb67c16a3f0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 >> 172.21.15.123:0/3488555693 conn(0x7fb67c0cd020 msgr2=0x7fb67c0ca3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:26.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 shutdown_connections 2023-10-06T04:32:26.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.456+0000 7fb684552700 1 -- 172.21.15.123:0/3488555693 wait complete. 2023-10-06T04:32:26.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-10-06T04:32:26.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.596+0000 7f62acd5d700 1 Processor -- start 2023-10-06T04:32:26.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.597+0000 7f62acd5d700 1 -- start start 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.597+0000 7f62acd5d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f62a806e9f0 0x7f62a8161200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.597+0000 7f62acd5d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a806a4e0 con 0x7f62a813fa70 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.597+0000 7f62acd5d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62a80c8ec0 con 0x7f62a80c8a10 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.597+0000 7f62acd5d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62a80c91a0 con 0x7f62a806e9f0 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a659c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f62a813fa70 0x7f62a80c8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40370/0 (socket says 172.21.15.123:40370) 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a659c700 1 -- 172.21.15.123:0/1349537899 learned_addr learned my addr 172.21.15.123:0/1349537899 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 229283677 0 0) 0x7f62a806a4e0 con 0x7f62a813fa70 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a6d9d700 1 --2- 172.21.15.123:0/1349537899 >> v2:172.21.15.177:3300/0 conn(0x7f62a806e9f0 0x7f62a8161200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6290003570 con 0x7f62a813fa70 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4189360701 0 0) 0x7f6290003570 con 0x7f62a813fa70 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 >> v2:172.21.15.177:3300/0 conn(0x7f62a806e9f0 msgr2=0x7f62a8161200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 --2- 172.21.15.123:0/1349537899 >> v2:172.21.15.177:3300/0 conn(0x7f62a806e9f0 0x7f62a8161200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 >> v1:172.21.15.177:6789/0 conn(0x7f62a80c8a10 legacy=0x7f62a806e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80c9480 con 0x7f62a813fa70 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f629c001470 con 0x7f62a813fa70 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 >> v1:172.21.15.123:6789/0 conn(0x7f62a813fa70 legacy=0x7f62a80c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f6290003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6290004810 0x7f6290004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62900051c0 0x7f629000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80c91a0 con 0x7f6290003a60 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f629000c0d0 con 0x7f62900051c0 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f629000c290 con 0x7f6290004810 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a659c700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f6290003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a5d9b700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6290004810 0x7f6290004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.598+0000 7f62a6d9d700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62900051c0 0x7f629000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a659c700 1 -- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6290004810 msgr2=0x7f6290004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a659c700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6290004810 0x7f6290004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a659c700 1 -- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62900051c0 msgr2=0x7f629000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a659c700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62900051c0 0x7f629000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a659c700 1 -- 172.21.15.123:0/1349537899 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6290003570 con 0x7f6290003a60 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a5d9b700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6290004810 0x7f6290004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:26.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a6d9d700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62900051c0 0x7f629000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a6d9d700 1 -- 172.21.15.123:0/1349537899 reap_dead start 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a659c700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f6290003e70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c3c0 tx=0x7f629c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb0329814d2a6304 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c00cc90 con 0x7f6290003a60 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c00cdf0 con 0x7f6290003a60 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62a559a700 1 -- 172.21.15.123:0/1349537899 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c005b50 con 0x7f6290003a60 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62acd5d700 1 -- 172.21.15.123:0/1349537899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 msgr2=0x7f6290003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.599+0000 7f62acd5d700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f6290003e70 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c3c0 tx=0x7f629c001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.600+0000 7f62acd5d700 1 -- 172.21.15.123:0/1349537899 shutdown_connections 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.600+0000 7f62acd5d700 1 --2- 172.21.15.123:0/1349537899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f6290003e70 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.600+0000 7f62acd5d700 1 -- 172.21.15.123:0/1349537899 >> 172.21.15.123:0/1349537899 conn(0x7f62a80cd020 msgr2=0x7f62a80cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:26.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.600+0000 7f62acd5d700 1 -- 172.21.15.123:0/1349537899 shutdown_connections 2023-10-06T04:32:26.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.600+0000 7f62acd5d700 1 -- 172.21.15.123:0/1349537899 wait complete. 2023-10-06T04:32:26.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 Processor -- start 2023-10-06T04:32:26.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 -- start start 2023-10-06T04:32:26.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f62a81533f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62a80ac710 0x7f62a813db70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62a80c2d90 0x7f62a80c3200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a8153930 con 0x7f6290003a60 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80c3740 con 0x7f62a80c2d90 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.601+0000 7f62acd5d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62a80ad830 con 0x7f62a80ac710 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f62a81533f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f62a81533f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35786/0 (socket says 172.21.15.123:35786) 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 -- 172.21.15.123:0/2502750513 learned_addr learned my addr 172.21.15.123:0/2502750513 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a5d9b700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62a80ac710 0x7f62a813db70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a6d9d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62a80c2d90 0x7f62a80c3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 -- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62a80ac710 msgr2=0x7f62a813db70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62a80ac710 0x7f62a813db70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 -- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62a80c2d90 msgr2=0x7f62a80c3200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62a80c2d90 0x7f62a80c3200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 -- 172.21.15.123:0/2502750513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a80acc20 con 0x7f6290003a60 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a6d9d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62a80c2d90 0x7f62a80c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62a659c700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f62a81533f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c9b0 tx=0x7f629c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62967fc700 1 -- 172.21.15.123:0/2502750513 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c000bc0 con 0x7f6290003a60 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62967fc700 1 -- 172.21.15.123:0/2502750513 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f629c0070d0 con 0x7f6290003a60 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62a80acf00 con 0x7f6290003a60 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62967fc700 1 -- 172.21.15.123:0/2502750513 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f629c007490 con 0x7f6290003a60 2023-10-06T04:32:26.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.602+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62a80ad4c0 con 0x7f6290003a60 2023-10-06T04:32:26.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.603+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62a806d240 con 0x7f6290003a60 2023-10-06T04:32:26.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.604+0000 7f62967fc700 1 -- 172.21.15.123:0/2502750513 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f629c028460 con 0x7f6290003a60 2023-10-06T04:32:26.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.605+0000 7f62967fc700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6288066f60 0x7f6288069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:26.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.605+0000 7f62967fc700 1 -- 172.21.15.123:0/2502750513 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f629c086810 con 0x7f6290003a60 2023-10-06T04:32:26.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.605+0000 7f62a5d9b700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6288066f60 0x7f6288069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:26.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.605+0000 7f62a5d9b700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6288066f60 0x7f6288069410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6298007fd0 tx=0x7f6298009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:26.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.608+0000 7f62967fc700 1 -- 172.21.15.123:0/2502750513 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f629c053880 con 0x7f6290003a60 2023-10-06T04:32:26.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:26.832+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f62a804bb80 con 0x7f6290003a60 2023-10-06T04:32:27.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.457+0000 7f62967fc700 1 -- 172.21.15.123:0/2502750513 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v17) v1 ==== 122+0+0 (secure 0 0 0) 0x7f629c017070 con 0x7f6290003a60 2023-10-06T04:32:27.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.459+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6288066f60 msgr2=0x7f6288069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.459+0000 7f62acd5d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6288066f60 0x7f6288069410 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f6298007fd0 tx=0x7f6298009040 comp rx=0 tx=0).stop 2023-10-06T04:32:27.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.459+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 msgr2=0x7f62a81533f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.459+0000 7f62acd5d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f62a81533f0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f629c00c9b0 tx=0x7f629c015000 comp rx=0 tx=0).stop 2023-10-06T04:32:27.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 shutdown_connections 2023-10-06T04:32:27.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6288066f60 0x7f6288069410 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6290003a60 0x7f62a81533f0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f62a80c2d90 0x7f62a80c3200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 --2- 172.21.15.123:0/2502750513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f62a80ac710 0x7f62a813db70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 >> 172.21.15.123:0/2502750513 conn(0x7f62a80cd020 msgr2=0x7f62a813e880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:27.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 shutdown_connections 2023-10-06T04:32:27.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.460+0000 7f62acd5d700 1 -- 172.21.15.123:0/2502750513 wait complete. 2023-10-06T04:32:27.475 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:27.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.590+0000 7ff3b8635700 1 Processor -- start 2023-10-06T04:32:27.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.591+0000 7ff3b8635700 1 -- start start 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.591+0000 7ff3b8635700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b013f600 0x7ff3b006b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.591+0000 7ff3b8635700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff3b006b990 0x7ff3b015d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.591+0000 7ff3b8635700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3b006be00 con 0x7ff3b015d550 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.591+0000 7ff3b8635700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3b00690a0 con 0x7ff3b006b990 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.591+0000 7ff3b8635700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b015dd40 con 0x7ff3b013f600 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff3b006b990 0x7ff3b015d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff3b006b990 0x7ff3b015d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35792/0 (socket says 172.21.15.123:35792) 2023-10-06T04:32:27.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b6bd2700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff3b015d550 0x7ff3b006cd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40382/0 (socket says 172.21.15.123:40382) 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 -- 172.21.15.123:0/1959575703 learned_addr learned my addr 172.21.15.123:0/1959575703 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b63d1700 1 --2- 172.21.15.123:0/1959575703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b013f600 0x7ff3b006b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 -- 172.21.15.123:0/1959575703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b013f600 msgr2=0x7ff3b006b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 --2- 172.21.15.123:0/1959575703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b013f600 0x7ff3b006b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 -- 172.21.15.123:0/1959575703 >> v1:172.21.15.123:6789/0 conn(0x7ff3b015d550 legacy=0x7ff3b006cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 -- 172.21.15.123:0/1959575703 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3b0068770 con 0x7ff3b006b990 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b63d1700 1 --2- 172.21.15.123:0/1959575703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b013f600 0x7ff3b006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b53cf700 1 -- 172.21.15.123:0/1959575703 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3328026891 0 0) 0x7ff3b006be00 con 0x7ff3b015d550 2023-10-06T04:32:27.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b5bd0700 1 --2- 172.21.15.123:0/1959575703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b015d010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0099b0 tx=0x7ff3ac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b3138de4c550bd5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b53cf700 1 -- 172.21.15.123:0/1959575703 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00b860 con 0x7ff3b006b990 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b53cf700 1 -- 172.21.15.123:0/1959575703 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3ac00b9c0 con 0x7ff3b006b990 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.592+0000 7ff3b53cf700 1 -- 172.21.15.123:0/1959575703 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3ac00bd60 con 0x7ff3b006b990 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 -- 172.21.15.123:0/1959575703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 msgr2=0x7ff3b015d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 --2- 172.21.15.123:0/1959575703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b015d010 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0099b0 tx=0x7ff3ac009250 comp rx=0 tx=0).stop 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 -- 172.21.15.123:0/1959575703 shutdown_connections 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 --2- 172.21.15.123:0/1959575703 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b013f600 0x7ff3b006b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 --2- 172.21.15.123:0/1959575703 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b015d010 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 -- 172.21.15.123:0/1959575703 >> 172.21.15.123:0/1959575703 conn(0x7ff3b00ce0c0 msgr2=0x7ff3b00ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:27.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 -- 172.21.15.123:0/1959575703 shutdown_connections 2023-10-06T04:32:27.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.593+0000 7ff3b8635700 1 -- 172.21.15.123:0/1959575703 wait complete. 2023-10-06T04:32:27.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.594+0000 7ff3b8635700 1 Processor -- start 2023-10-06T04:32:27.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.594+0000 7ff3b8635700 1 -- start start 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b8635700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b00c2d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b8635700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3b013f600 0x7ff3b014d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b8635700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b015d550 0x7ff3b00c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b8635700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b014daf0 con 0x7ff3b006b990 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b8635700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c2890 con 0x7ff3b013f600 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b8635700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3b00c1c80 con 0x7ff3b015d550 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b00c2d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b00c2d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35806/0 (socket says 172.21.15.123:35806) 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 -- 172.21.15.123:0/3977997055 learned_addr learned my addr 172.21.15.123:0/3977997055 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b6bd2700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b015d550 0x7ff3b00c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b5bd0700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3b013f600 0x7ff3b014d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 -- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b015d550 msgr2=0x7ff3b00c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b015d550 0x7ff3b00c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 -- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3b013f600 msgr2=0x7ff3b014d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3b013f600 0x7ff3b014d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 -- 172.21.15.123:0/3977997055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3ac0094f0 con 0x7ff3b006b990 2023-10-06T04:32:27.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b5bd0700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3b013f600 0x7ff3b014d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.595+0000 7ff3b63d1700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b00c2d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c830 tx=0x7ff3a0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.596+0000 7ff3a67fc700 1 -- 172.21.15.123:0/3977997055 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0011600 con 0x7ff3b006b990 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.596+0000 7ff3a67fc700 1 -- 172.21.15.123:0/3977997055 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3a0011760 con 0x7ff3b006b990 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.596+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3b00c1f60 con 0x7ff3b006b990 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.596+0000 7ff3a67fc700 1 -- 172.21.15.123:0/3977997055 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3a0011b20 con 0x7ff3b006b990 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.596+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3b00c2520 con 0x7ff3b006b990 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.597+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3b00c2b70 con 0x7ff3b006b990 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.597+0000 7ff3a67fc700 1 -- 172.21.15.123:0/3977997055 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff3a000a3e0 con 0x7ff3b006b990 2023-10-06T04:32:27.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.598+0000 7ff3a67fc700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.598+0000 7ff3a67fc700 1 -- 172.21.15.123:0/3977997055 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff3a0084cc0 con 0x7ff3b006b990 2023-10-06T04:32:27.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.598+0000 7ff3b5bd0700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.598+0000 7ff3b5bd0700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff394066f30 0x7ff3940693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0099b0 tx=0x7ff3ac017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:27.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.601+0000 7ff3a67fc700 1 -- 172.21.15.123:0/3977997055 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff3a0051e50 con 0x7ff3b006b990 2023-10-06T04:32:27.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.824+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff3b0068a50 con 0x7ff3b006b990 2023-10-06T04:32:27.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.825+0000 7ff3a67fc700 1 -- 172.21.15.123:0/3977997055 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4609 (secure 0 0 0) 0x7ff3a0055470 con 0x7ff3b006b990 2023-10-06T04:32:27.828 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:27.828 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2836433084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2836433084},{"type":"v1","addr":"172.21.15.177:6834","nonce":2836433084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3003404474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3003404474},{"type":"v1","addr":"172.21.15.123:6836","nonce":3003404474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1646131544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1646131544},{"type":"v1","addr":"172.21.15.123:6837","nonce":1646131544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:27.453433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/1099021797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1099021797},{"type":"v1","addr":"172.21.15.177:6835","nonce":1099021797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:32:27.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff394066f30 msgr2=0x7ff3940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff394066f30 0x7ff3940693e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff3ac0099b0 tx=0x7ff3ac017000 comp rx=0 tx=0).stop 2023-10-06T04:32:27.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 msgr2=0x7ff3b00c2d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b00c2d20 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff3a000c830 tx=0x7ff3a0008040 comp rx=0 tx=0).stop 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 shutdown_connections 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff394066f30 0x7ff3940693e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff3b006b990 0x7ff3b00c2d20 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff3b015d550 0x7ff3b00c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 --2- 172.21.15.123:0/3977997055 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff3b013f600 0x7ff3b014d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 >> 172.21.15.123:0/3977997055 conn(0x7ff3b00ce0c0 msgr2=0x7ff3b00ca460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:27.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 shutdown_connections 2023-10-06T04:32:27.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.827+0000 7ff3b8635700 1 -- 172.21.15.123:0/3977997055 wait complete. 2023-10-06T04:32:27.833 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 17 2023-10-06T04:32:27.841 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:27.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.957+0000 7f9e0d615700 1 Processor -- start 2023-10-06T04:32:27.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.958+0000 7f9e0d615700 1 -- start start 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.958+0000 7f9e0d615700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9e081593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.958+0000 7f9e0d615700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e08069a30 con 0x7f9e08068c90 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.958+0000 7f9e0d615700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e0806a890 con 0x7f9e080688c0 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.958+0000 7f9e0d615700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08159930 con 0x7f9e08158f70 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.958+0000 7f9e067fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9e08068c90 0x7f9e0806e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40394/0 (socket says 172.21.15.123:40394) 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.958+0000 7f9e067fc700 1 -- 172.21.15.123:0/3493425420 learned_addr learned my addr 172.21.15.123:0/3493425420 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e077fe700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9e081593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3669125395 0 0) 0x7f9e08069a30 con 0x7f9e08068c90 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9df0003570 con 0x7f9e08068c90 2023-10-06T04:32:27.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2202770620 0 0) 0x7f9e0806a890 con 0x7f9e080688c0 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e08069a30 con 0x7f9e080688c0 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 953901582 0 0) 0x7f9df0003570 con 0x7f9e08068c90 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 msgr2=0x7f9e081593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9e081593f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 >> v1:172.21.15.177:6789/0 conn(0x7f9e080688c0 legacy=0x7f9e080c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e08069c80 con 0x7f9e08068c90 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f9df8001470 con 0x7f9e08068c90 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e077fe700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9e081593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 >> v1:172.21.15.123:6789/0 conn(0x7f9e08068c90 legacy=0x7f9e0806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9df0004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df0004810 0x7f9df0004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9df00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e08159930 con 0x7f9df0003c20 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9df000bb40 con 0x7f9df0004810 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9df000bd00 con 0x7f9e08158f70 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e06ffd700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9df0004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e067fc700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df0004810 0x7f9df0004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e077fe700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9df00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e06ffd700 1 -- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 msgr2=0x7f9df00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e06ffd700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9df00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e06ffd700 1 -- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df0004810 msgr2=0x7f9df0004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.959+0000 7f9e06ffd700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9df0004810 0x7f9df0004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e06ffd700 1 -- 172.21.15.123:0/3493425420 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e00009a20 con 0x7f9df0003c20 2023-10-06T04:32:27.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e077fe700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e08158f70 0x7f9df00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e06ffd700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9df0004010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc001a10 tx=0x7f9dfc007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395c33fec34342d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00c5d0 con 0x7f9df0003c20 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc00c730 con 0x7f9df0003c20 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e05ffb700 1 -- 172.21.15.123:0/3493425420 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00cad0 con 0x7f9df0003c20 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e0d615700 1 -- 172.21.15.123:0/3493425420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 msgr2=0x7f9df0004010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e0d615700 1 --2- 172.21.15.123:0/3493425420 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9df0004010 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc001a10 tx=0x7f9dfc007d30 comp rx=0 tx=0).stop 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e077fe700 1 -- 172.21.15.123:0/3493425420 reap_dead start 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e0d615700 1 -- 172.21.15.123:0/3493425420 shutdown_connections 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e0d615700 1 -- 172.21.15.123:0/3493425420 >> 172.21.15.123:0/3493425420 conn(0x7f9e080c9c90 msgr2=0x7f9e0806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e0d615700 1 -- 172.21.15.123:0/3493425420 shutdown_connections 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.960+0000 7f9e0d615700 1 -- 172.21.15.123:0/3493425420 wait complete. 2023-10-06T04:32:27.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 Processor -- start 2023-10-06T04:32:27.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 -- start start 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9e0816d780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e080c38c0 0x7f9e080c3d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e0815c070 0x7f9e0815c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080c4f50 con 0x7f9df0003c20 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e0815ca00 con 0x7f9e0815c070 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e0d615700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e080c4340 con 0x7f9e080c38c0 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9e0816d780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9e0816d780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35832/0 (socket says 172.21.15.123:35832) 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.962+0000 7f9e06ffd700 1 -- 172.21.15.123:0/3290392979 learned_addr learned my addr 172.21.15.123:0/3290392979 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:27.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e067fc700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e080c38c0 0x7f9e080c3d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e077fe700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e0815c070 0x7f9e0815c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e06ffd700 1 -- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e080c38c0 msgr2=0x7f9e080c3d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e06ffd700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e080c38c0 0x7f9e080c3d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e06ffd700 1 -- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e0815c070 msgr2=0x7f9e0815c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e06ffd700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e0815c070 0x7f9e0815c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e06ffd700 1 -- 172.21.15.123:0/3290392979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e080c4620 con 0x7f9df0003c20 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e06ffd700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9e0816d780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc000c00 tx=0x7f9dfc015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9deeffd700 1 -- 172.21.15.123:0/3290392979 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc00c5d0 con 0x7f9df0003c20 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9deeffd700 1 -- 172.21.15.123:0/3290392979 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9dfc000fe0 con 0x7f9df0003c20 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e080c4900 con 0x7f9df0003c20 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9deeffd700 1 -- 172.21.15.123:0/3290392979 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9dfc01e650 con 0x7f9df0003c20 2023-10-06T04:32:27.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.963+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e0816dcc0 con 0x7f9df0003c20 2023-10-06T04:32:27.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.964+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e080c5230 con 0x7f9df0003c20 2023-10-06T04:32:27.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.964+0000 7f9deeffd700 1 -- 172.21.15.123:0/3290392979 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9dfc019070 con 0x7f9df0003c20 2023-10-06T04:32:27.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.965+0000 7f9deeffd700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9df4066f30 0x7f9df40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:27.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.965+0000 7f9deeffd700 1 -- 172.21.15.123:0/3290392979 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f9dfc085b30 con 0x7f9df0003c20 2023-10-06T04:32:27.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.965+0000 7f9e067fc700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9df4066f30 0x7f9df40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:27.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.965+0000 7f9e067fc700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9df4066f30 0x7f9df40693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000e50 tx=0x7f9e080ca7a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:27.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:27.969+0000 7f9deeffd700 1 -- 172.21.15.123:0/3290392979 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9dfc052c40 con 0x7f9df0003c20 2023-10-06T04:32:28.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.151+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e08069f60 con 0x7f9df0003c20 2023-10-06T04:32:28.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.152+0000 7f9deeffd700 1 -- 172.21.15.123:0/3290392979 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+17010 (secure 0 0 0) 0x7f9dfc017090 con 0x7f9df0003c20 2023-10-06T04:32:28.155 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:28.155 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":34,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:25.247769+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9df4066f30 msgr2=0x7f9df40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9df4066f30 0x7f9df40693e0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f9df8000e50 tx=0x7f9e080ca7a0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 msgr2=0x7f9e0816d780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9e0816d780 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7f9dfc000c00 tx=0x7f9dfc015000 comp rx=0 tx=0).stop 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 shutdown_connections 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9df4066f30 0x7f9df40693e0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9df0003c20 0x7f9e0816d780 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9e0815c070 0x7f9e0815c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.157 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 --2- 172.21.15.123:0/3290392979 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9e080c38c0 0x7f9e080c3d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.158 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 >> 172.21.15.123:0/3290392979 conn(0x7f9e080c9c90 msgr2=0x7f9e08158130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:28.158 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 shutdown_connections 2023-10-06T04:32:28.158 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.154+0000 7f9e0d615700 1 -- 172.21.15.123:0/3290392979 wait complete. 2023-10-06T04:32:28.169 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:32:28.169 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:32:28.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.283+0000 7f2e52b7f700 1 Processor -- start 2023-10-06T04:32:28.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.283+0000 7f2e52b7f700 1 -- start start 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e52b7f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e52b7f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2e4c15b290 0x7f2e4c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e52b7f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2e4c157910 con 0x7f2e4c06cd60 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e52b7f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2e4c06d240 con 0x7f2e4c15b290 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e52b7f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c06d520 con 0x7f2e4c156f60 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e4bfff700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2e4c15b290 0x7f2e4c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e5091b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e5111c700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f2e4c06cd60 0x7f2e4c157da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:45182/0 (socket says 172.21.15.123:45182) 2023-10-06T04:32:28.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.284+0000 7f2e5111c700 1 -- 172.21.15.123:0/3624600380 learned_addr learned my addr 172.21.15.123:0/3624600380 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4bfff700 1 -- 172.21.15.123:0/3624600380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 msgr2=0x7f2e4c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4bfff700 1 --2- 172.21.15.123:0/3624600380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4bfff700 1 -- 172.21.15.123:0/3624600380 >> v1:172.21.15.177:6789/0 conn(0x7f2e4c06cd60 legacy=0x7f2e4c157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4bfff700 1 -- 172.21.15.123:0/3624600380 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e4c06d800 con 0x7f2e4c15b290 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4bfff700 1 --2- 172.21.15.123:0/3624600380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 0x7f2e4c0ad730 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2e380098e0 tx=0x7f2e380091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ce68be32439df4aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4b7fe700 1 -- 172.21.15.123:0/3624600380 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800b640 con 0x7f2e4c15b290 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4b7fe700 1 -- 172.21.15.123:0/3624600380 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e3800b7a0 con 0x7f2e4c15b290 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e4b7fe700 1 -- 172.21.15.123:0/3624600380 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e3800bb40 con 0x7f2e4c15b290 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3624600380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 msgr2=0x7f2e4c0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.285+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3624600380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 0x7f2e4c0ad730 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f2e380098e0 tx=0x7f2e380091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.286+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3624600380 shutdown_connections 2023-10-06T04:32:28.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.286+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3624600380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.286+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3624600380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 0x7f2e4c0ad730 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.286+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3624600380 >> 172.21.15.123:0/3624600380 conn(0x7f2e4c0ce0c0 msgr2=0x7f2e4c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:28.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.286+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3624600380 shutdown_connections 2023-10-06T04:32:28.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.286+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3624600380 wait complete. 2023-10-06T04:32:28.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 Processor -- start 2023-10-06T04:32:28.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 -- start start 2023-10-06T04:32:28.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4c06cd60 0x7f2e4c0c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c159990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 0x7f2e4c16acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c0c2420 con 0x7f2e4c06cd60 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c168e60 con 0x7f2e4c15b290 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.287+0000 7f2e52b7f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2e4c15a220 con 0x7f2e4c156f60 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4c06cd60 0x7f2e4c0c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4c06cd60 0x7f2e4c0c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35846/0 (socket says 172.21.15.123:35846) 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 -- 172.21.15.123:0/3107256856 learned_addr learned my addr 172.21.15.123:0/3107256856 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5111c700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 0x7f2e4c16acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e4bfff700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c159990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 -- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 msgr2=0x7f2e4c159990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c159990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 -- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 msgr2=0x7f2e4c16acc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 0x7f2e4c16acc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 -- 172.21.15.123:0/3107256856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2e38009420 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e5091b700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4c06cd60 0x7f2e4c0c45a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000ec80 tx=0x7f2e400091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e48ff9700 1 -- 172.21.15.123:0/3107256856 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000c850 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e48ff9700 1 -- 172.21.15.123:0/3107256856 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2e4000c9b0 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e48ff9700 1 -- 172.21.15.123:0/3107256856 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2e4000cd50 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2e4c0c1af0 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.288+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2e4c0c20b0 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.289+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2e4c0c2700 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.290+0000 7f2e48ff9700 1 -- 172.21.15.123:0/3107256856 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2e40013680 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.290+0000 7f2e48ff9700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.290+0000 7f2e48ff9700 1 -- 172.21.15.123:0/3107256856 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f2e40084910 con 0x7f2e4c06cd60 2023-10-06T04:32:28.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.290+0000 7f2e4bfff700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.290+0000 7f2e4bfff700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2e30066f30 0x7f2e300693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e380093f0 tx=0x7f2e38017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:28.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.294+0000 7f2e48ff9700 1 -- 172.21.15.123:0/3107256856 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2e40051a20 con 0x7f2e4c06cd60 2023-10-06T04:32:28.414 INFO:tasks.ceph.mds.d.smithi177.stderr:2023-10-06T04:32:28.412+0000 7fb1edb68700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:32:28.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.471+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2e4c0c2560 con 0x7f2e30066f30 2023-10-06T04:32:28.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.480+0000 7f2e48ff9700 1 -- 172.21.15.123:0/3107256856 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311185 (secure 0 0 0) 0x7f2e3800b910 con 0x7f2e30066f30 2023-10-06T04:32:28.486 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:28.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2e30066f30 msgr2=0x7f2e300693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2e30066f30 0x7f2e300693e0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f2e380093f0 tx=0x7f2e38017000 comp rx=0 tx=0).stop 2023-10-06T04:32:28.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4c06cd60 msgr2=0x7f2e4c0c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4c06cd60 0x7f2e4c0c45a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f2e4000ec80 tx=0x7f2e400091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 shutdown_connections 2023-10-06T04:32:28.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2e30066f30 0x7f2e300693e0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2e4c06cd60 0x7f2e4c0c45a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2e4c15b290 0x7f2e4c16acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 --2- 172.21.15.123:0/3107256856 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2e4c156f60 0x7f2e4c159990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.483+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 >> 172.21.15.123:0/3107256856 conn(0x7f2e4c0ce0c0 msgr2=0x7f2e4c0b37e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.484+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 shutdown_connections 2023-10-06T04:32:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.484+0000 7f2e52b7f700 1 -- 172.21.15.123:0/3107256856 wait complete. 2023-10-06T04:32:28.490 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:32:28.498 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-06T04:32:26.899805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382452,"kb_used_data":4148,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592268,"statfs":{"total":773094113280,"available":770654482432,"internally_reserved":0,"allocated":4247552,"data_stored":2390349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.476096"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270484+0000","last_change":"2023-10-06T04:32:24.268479+0000","last_active":"2023-10-06T04:32:24.270484+0000","last_peered":"2023-10-06T04:32:24.270484+0000","last_clean":"2023-10-06T04:32:24.270484+0000","last_became_active":"2023-10-06T04:32:24.268175+0000","last_became_peered":"2023-10-06T04:32:24.268175+0000","last_unstale":"2023-10-06T04:32:24.270484+0000","last_undegraded":"2023-10-06T04:32:24.270484+0000","last_fullsized":"2023-10-06T04:32:24.270484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272058+0000","last_change":"2023-10-06T04:32:24.269331+0000","last_active":"2023-10-06T04:32:24.272058+0000","last_peered":"2023-10-06T04:32:24.272058+0000","last_clean":"2023-10-06T04:32:24.272058+0000","last_became_active":"2023-10-06T04:32:24.269096+0000","last_became_peered":"2023-10-06T04:32:24.269096+0000","last_unstale":"2023-10-06T04:32:24.272058+0000","last_undegraded":"2023-10-06T04:32:24.272058+0000","last_fullsized":"2023-10-06T04:32:24.272058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274016+0000","last_change":"2023-10-06T04:32:24.267252+0000","last_active":"2023-10-06T04:32:24.274016+0000","last_peered":"2023-10-06T04:32:24.274016+0000","last_clean":"2023-10-06T04:32:24.274016+0000","last_became_active":"2023-10-06T04:32:24.266928+0000","last_became_peered":"2023-10-06T04:32:24.266928+0000","last_unstale":"2023-10-06T04:32:24.274016+0000","last_undegraded":"2023-10-06T04:32:24.274016+0000","last_fullsized":"2023-10-06T04:32:24.274016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272415+0000","last_change":"2023-10-06T04:32:24.271555+0000","last_active":"2023-10-06T04:32:24.272415+0000","last_peered":"2023-10-06T04:32:24.272415+0000","last_clean":"2023-10-06T04:32:24.272415+0000","last_became_active":"2023-10-06T04:32:24.271271+0000","last_became_peered":"2023-10-06T04:32:24.271271+0000","last_unstale":"2023-10-06T04:32:24.272415+0000","last_undegraded":"2023-10-06T04:32:24.272415+0000","last_fullsized":"2023-10-06T04:32:24.272415+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.271957+0000","last_change":"2023-10-06T04:32:24.270442+0000","last_active":"2023-10-06T04:32:24.271957+0000","last_peered":"2023-10-06T04:32:24.271957+0000","last_clean":"2023-10-06T04:32:24.271957+0000","last_became_active":"2023-10-06T04:32:24.270131+0000","last_became_peered":"2023-10-06T04:32:24.270131+0000","last_unstale":"2023-10-06T04:32:24.271957+0000","last_undegraded":"2023-10-06T04:32:24.271957+0000","last_fullsized":"2023-10-06T04:32:24.271957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269930+0000","last_change":"2023-10-06T04:32:24.263272+0000","last_active":"2023-10-06T04:32:24.269930+0000","last_peered":"2023-10-06T04:32:24.269930+0000","last_clean":"2023-10-06T04:32:24.269930+0000","last_became_active":"2023-10-06T04:32:24.262984+0000","last_became_peered":"2023-10-06T04:32:24.262984+0000","last_unstale":"2023-10-06T04:32:24.269930+0000","last_undegraded":"2023-10-06T04:32:24.269930+0000","last_fullsized":"2023-10-06T04:32:24.269930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:48.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270902+0000","last_change":"2023-10-06T04:32:24.263913+0000","last_active":"2023-10-06T04:32:24.270902+0000","last_peered":"2023-10-06T04:32:24.270902+0000","last_clean":"2023-10-06T04:32:24.270902+0000","last_became_active":"2023-10-06T04:32:24.263358+0000","last_became_peered":"2023-10-06T04:32:24.263358+0000","last_unstale":"2023-10-06T04:32:24.270902+0000","last_undegraded":"2023-10-06T04:32:24.270902+0000","last_fullsized":"2023-10-06T04:32:24.270902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270266+0000","last_change":"2023-10-06T04:32:24.266865+0000","last_active":"2023-10-06T04:32:24.270266+0000","last_peered":"2023-10-06T04:32:24.270266+0000","last_clean":"2023-10-06T04:32:24.270266+0000","last_became_active":"2023-10-06T04:32:24.266456+0000","last_became_peered":"2023-10-06T04:32:24.266456+0000","last_unstale":"2023-10-06T04:32:24.270266+0000","last_undegraded":"2023-10-06T04:32:24.270266+0000","last_fullsized":"2023-10-06T04:32:24.270266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274060+0000","last_change":"2023-10-06T04:32:24.267567+0000","last_active":"2023-10-06T04:32:24.274060+0000","last_peered":"2023-10-06T04:32:24.274060+0000","last_clean":"2023-10-06T04:32:24.274060+0000","last_became_active":"2023-10-06T04:32:24.267326+0000","last_became_peered":"2023-10-06T04:32:24.267326+0000","last_unstale":"2023-10-06T04:32:24.274060+0000","last_undegraded":"2023-10-06T04:32:24.274060+0000","last_fullsized":"2023-10-06T04:32:24.274060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269694+0000","last_change":"2023-10-06T04:32:24.266439+0000","last_active":"2023-10-06T04:32:24.269694+0000","last_peered":"2023-10-06T04:32:24.269694+0000","last_clean":"2023-10-06T04:32:24.269694+0000","last_became_active":"2023-10-06T04:32:24.266172+0000","last_became_peered":"2023-10-06T04:32:24.266172+0000","last_unstale":"2023-10-06T04:32:24.269694+0000","last_undegraded":"2023-10-06T04:32:24.269694+0000","last_fullsized":"2023-10-06T04:32:24.269694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:07.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269612+0000","last_change":"2023-10-06T04:32:24.264976+0000","last_active":"2023-10-06T04:32:24.269612+0000","last_peered":"2023-10-06T04:32:24.269612+0000","last_clean":"2023-10-06T04:32:24.269612+0000","last_became_active":"2023-10-06T04:32:24.264647+0000","last_became_peered":"2023-10-06T04:32:24.264647+0000","last_unstale":"2023-10-06T04:32:24.269612+0000","last_undegraded":"2023-10-06T04:32:24.269612+0000","last_fullsized":"2023-10-06T04:32:24.269612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.268980+0000","last_change":"2023-10-06T04:32:24.262121+0000","last_active":"2023-10-06T04:32:24.268980+0000","last_peered":"2023-10-06T04:32:24.268980+0000","last_clean":"2023-10-06T04:32:24.268980+0000","last_became_active":"2023-10-06T04:32:24.261846+0000","last_became_peered":"2023-10-06T04:32:24.261846+0000","last_unstale":"2023-10-06T04:32:24.268980+0000","last_undegraded":"2023-10-06T04:32:24.268980+0000","last_fullsized":"2023-10-06T04:32:24.268980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270791+0000","last_change":"2023-10-06T04:32:24.267116+0000","last_active":"2023-10-06T04:32:24.270791+0000","last_peered":"2023-10-06T04:32:24.270791+0000","last_clean":"2023-10-06T04:32:24.270791+0000","last_became_active":"2023-10-06T04:32:24.266756+0000","last_became_peered":"2023-10-06T04:32:24.266756+0000","last_unstale":"2023-10-06T04:32:24.270791+0000","last_undegraded":"2023-10-06T04:32:24.270791+0000","last_fullsized":"2023-10-06T04:32:24.270791+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:26.457033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274722+0000","last_change":"2023-10-06T04:32:24.267554+0000","last_active":"2023-10-06T04:32:24.274722+0000","last_peered":"2023-10-06T04:32:24.274722+0000","last_clean":"2023-10-06T04:32:24.274722+0000","last_became_active":"2023-10-06T04:32:24.266198+0000","last_became_peered":"2023-10-06T04:32:24.266198+0000","last_unstale":"2023-10-06T04:32:24.274722+0000","last_undegraded":"2023-10-06T04:32:24.274722+0000","last_fullsized":"2023-10-06T04:32:24.274722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270938+0000","last_change":"2023-10-06T04:32:24.265234+0000","last_active":"2023-10-06T04:32:24.270938+0000","last_peered":"2023-10-06T04:32:24.270938+0000","last_clean":"2023-10-06T04:32:24.270938+0000","last_became_active":"2023-10-06T04:32:24.264892+0000","last_became_peered":"2023-10-06T04:32:24.264892+0000","last_unstale":"2023-10-06T04:32:24.270938+0000","last_undegraded":"2023-10-06T04:32:24.270938+0000","last_fullsized":"2023-10-06T04:32:24.270938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272961+0000","last_change":"2023-10-06T04:32:24.268696+0000","last_active":"2023-10-06T04:32:24.272961+0000","last_peered":"2023-10-06T04:32:24.272961+0000","last_clean":"2023-10-06T04:32:24.272961+0000","last_became_active":"2023-10-06T04:32:24.268430+0000","last_became_peered":"2023-10-06T04:32:24.268430+0000","last_unstale":"2023-10-06T04:32:24.272961+0000","last_undegraded":"2023-10-06T04:32:24.272961+0000","last_fullsized":"2023-10-06T04:32:24.272961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272716+0000","last_change":"2023-10-06T04:32:24.269272+0000","last_active":"2023-10-06T04:32:24.272716+0000","last_peered":"2023-10-06T04:32:24.272716+0000","last_clean":"2023-10-06T04:32:24.272716+0000","last_became_active":"2023-10-06T04:32:24.268793+0000","last_became_peered":"2023-10-06T04:32:24.268793+0000","last_unstale":"2023-10-06T04:32:24.272716+0000","last_undegraded":"2023-10-06T04:32:24.272716+0000","last_fullsized":"2023-10-06T04:32:24.272716+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273449+0000","last_change":"2023-10-06T04:32:24.268089+0000","last_active":"2023-10-06T04:32:24.273449+0000","last_peered":"2023-10-06T04:32:24.273449+0000","last_clean":"2023-10-06T04:32:24.273449+0000","last_became_active":"2023-10-06T04:32:24.267842+0000","last_became_peered":"2023-10-06T04:32:24.267842+0000","last_unstale":"2023-10-06T04:32:24.273449+0000","last_undegraded":"2023-10-06T04:32:24.273449+0000","last_fullsized":"2023-10-06T04:32:24.273449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:36.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269498+0000","last_change":"2023-10-06T04:32:24.263185+0000","last_active":"2023-10-06T04:32:24.269498+0000","last_peered":"2023-10-06T04:32:24.269498+0000","last_clean":"2023-10-06T04:32:24.269498+0000","last_became_active":"2023-10-06T04:32:24.262500+0000","last_became_peered":"2023-10-06T04:32:24.262500+0000","last_unstale":"2023-10-06T04:32:24.269498+0000","last_undegraded":"2023-10-06T04:32:24.269498+0000","last_fullsized":"2023-10-06T04:32:24.269498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:18.096304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272902+0000","last_change":"2023-10-06T04:32:24.266364+0000","last_active":"2023-10-06T04:32:24.272902+0000","last_peered":"2023-10-06T04:32:24.272902+0000","last_clean":"2023-10-06T04:32:24.272902+0000","last_became_active":"2023-10-06T04:32:24.266082+0000","last_became_peered":"2023-10-06T04:32:24.266082+0000","last_unstale":"2023-10-06T04:32:24.272902+0000","last_undegraded":"2023-10-06T04:32:24.272902+0000","last_fullsized":"2023-10-06T04:32:24.272902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269519+0000","last_change":"2023-10-06T04:32:24.265143+0000","last_active":"2023-10-06T04:32:24.269519+0000","last_peered":"2023-10-06T04:32:24.269519+0000","last_clean":"2023-10-06T04:32:24.269519+0000","last_became_active":"2023-10-06T04:32:24.264481+0000","last_became_peered":"2023-10-06T04:32:24.264481+0000","last_unstale":"2023-10-06T04:32:24.269519+0000","last_undegraded":"2023-10-06T04:32:24.269519+0000","last_fullsized":"2023-10-06T04:32:24.269519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:04.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269525+0000","last_change":"2023-10-06T04:32:24.262815+0000","last_active":"2023-10-06T04:32:24.269525+0000","last_peered":"2023-10-06T04:32:24.269525+0000","last_clean":"2023-10-06T04:32:24.269525+0000","last_became_active":"2023-10-06T04:32:24.262576+0000","last_became_peered":"2023-10-06T04:32:24.262576+0000","last_unstale":"2023-10-06T04:32:24.269525+0000","last_undegraded":"2023-10-06T04:32:24.269525+0000","last_fullsized":"2023-10-06T04:32:24.269525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272837+0000","last_change":"2023-10-06T04:32:24.269201+0000","last_active":"2023-10-06T04:32:24.272837+0000","last_peered":"2023-10-06T04:32:24.272837+0000","last_clean":"2023-10-06T04:32:24.272837+0000","last_became_active":"2023-10-06T04:32:24.268841+0000","last_became_peered":"2023-10-06T04:32:24.268841+0000","last_unstale":"2023-10-06T04:32:24.272837+0000","last_undegraded":"2023-10-06T04:32:24.272837+0000","last_fullsized":"2023-10-06T04:32:24.272837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270172+0000","last_change":"2023-10-06T04:32:24.264875+0000","last_active":"2023-10-06T04:32:24.270172+0000","last_peered":"2023-10-06T04:32:24.270172+0000","last_clean":"2023-10-06T04:32:24.270172+0000","last_became_active":"2023-10-06T04:32:24.264601+0000","last_became_peered":"2023-10-06T04:32:24.264601+0000","last_unstale":"2023-10-06T04:32:24.270172+0000","last_undegraded":"2023-10-06T04:32:24.270172+0000","last_fullsized":"2023-10-06T04:32:24.270172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:38.680988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273472+0000","last_change":"2023-10-06T04:32:24.268714+0000","last_active":"2023-10-06T04:32:24.273472+0000","last_peered":"2023-10-06T04:32:24.273472+0000","last_clean":"2023-10-06T04:32:24.273472+0000","last_became_active":"2023-10-06T04:32:24.268440+0000","last_became_peered":"2023-10-06T04:32:24.268440+0000","last_unstale":"2023-10-06T04:32:24.273472+0000","last_undegraded":"2023-10-06T04:32:24.273472+0000","last_fullsized":"2023-10-06T04:32:24.273472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:24.913639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270362+0000","last_change":"2023-10-06T04:32:24.266945+0000","last_active":"2023-10-06T04:32:24.270362+0000","last_peered":"2023-10-06T04:32:24.270362+0000","last_clean":"2023-10-06T04:32:24.270362+0000","last_became_active":"2023-10-06T04:32:24.266673+0000","last_became_peered":"2023-10-06T04:32:24.266673+0000","last_unstale":"2023-10-06T04:32:24.270362+0000","last_undegraded":"2023-10-06T04:32:24.270362+0000","last_fullsized":"2023-10-06T04:32:24.270362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273762+0000","last_change":"2023-10-06T04:32:24.270531+0000","last_active":"2023-10-06T04:32:24.273762+0000","last_peered":"2023-10-06T04:32:24.273762+0000","last_clean":"2023-10-06T04:32:24.273762+0000","last_became_active":"2023-10-06T04:32:24.270268+0000","last_became_peered":"2023-10-06T04:32:24.270268+0000","last_unstale":"2023-10-06T04:32:24.273762+0000","last_undegraded":"2023-10-06T04:32:24.273762+0000","last_fullsized":"2023-10-06T04:32:24.273762+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:34.859497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269547+0000","last_change":"2023-10-06T04:32:24.265317+0000","last_active":"2023-10-06T04:32:24.269547+0000","last_peered":"2023-10-06T04:32:24.269547+0000","last_clean":"2023-10-06T04:32:24.269547+0000","last_became_active":"2023-10-06T04:32:24.264377+0000","last_became_peered":"2023-10-06T04:32:24.264377+0000","last_unstale":"2023-10-06T04:32:24.269547+0000","last_undegraded":"2023-10-06T04:32:24.269547+0000","last_fullsized":"2023-10-06T04:32:24.269547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272464+0000","last_change":"2023-10-06T04:32:24.264442+0000","last_active":"2023-10-06T04:32:24.272464+0000","last_peered":"2023-10-06T04:32:24.272464+0000","last_clean":"2023-10-06T04:32:24.272464+0000","last_became_active":"2023-10-06T04:32:24.264148+0000","last_became_peered":"2023-10-06T04:32:24.264148+0000","last_unstale":"2023-10-06T04:32:24.272464+0000","last_undegraded":"2023-10-06T04:32:24.272464+0000","last_fullsized":"2023-10-06T04:32:24.272464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273792+0000","last_change":"2023-10-06T04:32:24.270800+0000","last_active":"2023-10-06T04:32:24.273792+0000","last_peered":"2023-10-06T04:32:24.273792+0000","last_clean":"2023-10-06T04:32:24.273792+0000","last_became_active":"2023-10-06T04:32:24.270560+0000","last_became_peered":"2023-10-06T04:32:24.270560+0000","last_unstale":"2023-10-06T04:32:24.273792+0000","last_undegraded":"2023-10-06T04:32:24.273792+0000","last_fullsized":"2023-10-06T04:32:24.273792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274625+0000","last_change":"2023-10-06T04:32:24.271609+0000","last_active":"2023-10-06T04:32:24.274625+0000","last_peered":"2023-10-06T04:32:24.274625+0000","last_clean":"2023-10-06T04:32:24.274625+0000","last_became_active":"2023-10-06T04:32:24.271354+0000","last_became_peered":"2023-10-06T04:32:24.271354+0000","last_unstale":"2023-10-06T04:32:24.274625+0000","last_undegraded":"2023-10-06T04:32:24.274625+0000","last_fullsized":"2023-10-06T04:32:24.274625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270797+0000","last_change":"2023-10-06T04:32:24.263719+0000","last_active":"2023-10-06T04:32:24.270797+0000","last_peered":"2023-10-06T04:32:24.270797+0000","last_clean":"2023-10-06T04:32:24.270797+0000","last_became_active":"2023-10-06T04:32:24.263442+0000","last_became_peered":"2023-10-06T04:32:24.263442+0000","last_unstale":"2023-10-06T04:32:24.270797+0000","last_undegraded":"2023-10-06T04:32:24.270797+0000","last_fullsized":"2023-10-06T04:32:24.270797+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:40.811547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274529+0000","last_change":"2023-10-06T04:32:24.270093+0000","last_active":"2023-10-06T04:32:24.274529+0000","last_peered":"2023-10-06T04:32:24.274529+0000","last_clean":"2023-10-06T04:32:24.274529+0000","last_became_active":"2023-10-06T04:32:24.269851+0000","last_became_peered":"2023-10-06T04:32:24.269851+0000","last_unstale":"2023-10-06T04:32:24.274529+0000","last_undegraded":"2023-10-06T04:32:24.274529+0000","last_fullsized":"2023-10-06T04:32:24.274529+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446192+0000","last_change":"2023-10-06T04:32:23.441194+0000","last_active":"2023-10-06T04:32:24.446192+0000","last_peered":"2023-10-06T04:32:24.446192+0000","last_clean":"2023-10-06T04:32:24.446192+0000","last_became_active":"2023-10-06T04:32:23.440849+0000","last_became_peered":"2023-10-06T04:32:23.440849+0000","last_unstale":"2023-10-06T04:32:24.446192+0000","last_undegraded":"2023-10-06T04:32:24.446192+0000","last_fullsized":"2023-10-06T04:32:24.446192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:59.829363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270962+0000","last_change":"2023-10-06T04:32:24.267279+0000","last_active":"2023-10-06T04:32:24.270962+0000","last_peered":"2023-10-06T04:32:24.270962+0000","last_clean":"2023-10-06T04:32:24.270962+0000","last_became_active":"2023-10-06T04:32:24.267016+0000","last_became_peered":"2023-10-06T04:32:24.267016+0000","last_unstale":"2023-10-06T04:32:24.270962+0000","last_undegraded":"2023-10-06T04:32:24.270962+0000","last_fullsized":"2023-10-06T04:32:24.270962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.443655+0000","last_change":"2023-10-06T04:32:23.450031+0000","last_active":"2023-10-06T04:32:24.443655+0000","last_peered":"2023-10-06T04:32:24.443655+0000","last_clean":"2023-10-06T04:32:24.443655+0000","last_became_active":"2023-10-06T04:32:23.449717+0000","last_became_peered":"2023-10-06T04:32:23.449717+0000","last_unstale":"2023-10-06T04:32:24.443655+0000","last_undegraded":"2023-10-06T04:32:24.443655+0000","last_fullsized":"2023-10-06T04:32:24.443655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:19.596388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274702+0000","last_change":"2023-10-06T04:32:24.267302+0000","last_active":"2023-10-06T04:32:24.274702+0000","last_peered":"2023-10-06T04:32:24.274702+0000","last_clean":"2023-10-06T04:32:24.274702+0000","last_became_active":"2023-10-06T04:32:24.266976+0000","last_became_peered":"2023-10-06T04:32:24.266976+0000","last_unstale":"2023-10-06T04:32:24.274702+0000","last_undegraded":"2023-10-06T04:32:24.274702+0000","last_fullsized":"2023-10-06T04:32:24.274702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.443427+0000","last_change":"2023-10-06T04:32:23.447029+0000","last_active":"2023-10-06T04:32:24.443427+0000","last_peered":"2023-10-06T04:32:24.443427+0000","last_clean":"2023-10-06T04:32:24.443427+0000","last_became_active":"2023-10-06T04:32:23.446659+0000","last_became_peered":"2023-10-06T04:32:23.446659+0000","last_unstale":"2023-10-06T04:32:24.443427+0000","last_undegraded":"2023-10-06T04:32:24.443427+0000","last_fullsized":"2023-10-06T04:32:24.443427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:13.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270165+0000","last_change":"2023-10-06T04:32:24.265927+0000","last_active":"2023-10-06T04:32:24.270165+0000","last_peered":"2023-10-06T04:32:24.270165+0000","last_clean":"2023-10-06T04:32:24.270165+0000","last_became_active":"2023-10-06T04:32:24.265668+0000","last_became_peered":"2023-10-06T04:32:24.265668+0000","last_unstale":"2023-10-06T04:32:24.270165+0000","last_undegraded":"2023-10-06T04:32:24.270165+0000","last_fullsized":"2023-10-06T04:32:24.270165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.442929+0000","last_change":"2023-10-06T04:32:23.437810+0000","last_active":"2023-10-06T04:32:24.442929+0000","last_peered":"2023-10-06T04:32:24.442929+0000","last_clean":"2023-10-06T04:32:24.442929+0000","last_became_active":"2023-10-06T04:32:23.437533+0000","last_became_peered":"2023-10-06T04:32:23.437533+0000","last_unstale":"2023-10-06T04:32:24.442929+0000","last_undegraded":"2023-10-06T04:32:24.442929+0000","last_fullsized":"2023-10-06T04:32:24.442929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:46.149228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.268930+0000","last_change":"2023-10-06T04:32:24.261934+0000","last_active":"2023-10-06T04:32:24.268930+0000","last_peered":"2023-10-06T04:32:24.268930+0000","last_clean":"2023-10-06T04:32:24.268930+0000","last_became_active":"2023-10-06T04:32:24.261681+0000","last_became_peered":"2023-10-06T04:32:24.261681+0000","last_unstale":"2023-10-06T04:32:24.268930+0000","last_undegraded":"2023-10-06T04:32:24.268930+0000","last_fullsized":"2023-10-06T04:32:24.268930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440007+0000","last_change":"2023-10-06T04:32:23.443656+0000","last_active":"2023-10-06T04:32:24.440007+0000","last_peered":"2023-10-06T04:32:24.440007+0000","last_clean":"2023-10-06T04:32:24.440007+0000","last_became_active":"2023-10-06T04:32:23.443293+0000","last_became_peered":"2023-10-06T04:32:23.443293+0000","last_unstale":"2023-10-06T04:32:24.440007+0000","last_undegraded":"2023-10-06T04:32:24.440007+0000","last_fullsized":"2023-10-06T04:32:24.440007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270922+0000","last_change":"2023-10-06T04:32:24.264104+0000","last_active":"2023-10-06T04:32:24.270922+0000","last_peered":"2023-10-06T04:32:24.270922+0000","last_clean":"2023-10-06T04:32:24.270922+0000","last_became_active":"2023-10-06T04:32:24.263651+0000","last_became_peered":"2023-10-06T04:32:24.263651+0000","last_unstale":"2023-10-06T04:32:24.270922+0000","last_undegraded":"2023-10-06T04:32:24.270922+0000","last_fullsized":"2023-10-06T04:32:24.270922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:24.395312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446140+0000","last_change":"2023-10-06T04:32:23.451767+0000","last_active":"2023-10-06T04:32:24.446140+0000","last_peered":"2023-10-06T04:32:24.446140+0000","last_clean":"2023-10-06T04:32:24.446140+0000","last_became_active":"2023-10-06T04:32:23.451430+0000","last_became_peered":"2023-10-06T04:32:23.451430+0000","last_unstale":"2023-10-06T04:32:24.446140+0000","last_undegraded":"2023-10-06T04:32:24.446140+0000","last_fullsized":"2023-10-06T04:32:24.446140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274735+0000","last_change":"2023-10-06T04:32:24.267750+0000","last_active":"2023-10-06T04:32:24.274735+0000","last_peered":"2023-10-06T04:32:24.274735+0000","last_clean":"2023-10-06T04:32:24.274735+0000","last_became_active":"2023-10-06T04:32:24.265805+0000","last_became_peered":"2023-10-06T04:32:24.265805+0000","last_unstale":"2023-10-06T04:32:24.274735+0000","last_undegraded":"2023-10-06T04:32:24.274735+0000","last_fullsized":"2023-10-06T04:32:24.274735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251273+0000","last_change":"2023-10-06T04:32:23.449278+0000","last_active":"2023-10-06T04:32:24.251273+0000","last_peered":"2023-10-06T04:32:24.251273+0000","last_clean":"2023-10-06T04:32:24.251273+0000","last_became_active":"2023-10-06T04:32:23.449034+0000","last_became_peered":"2023-10-06T04:32:23.449034+0000","last_unstale":"2023-10-06T04:32:24.251273+0000","last_undegraded":"2023-10-06T04:32:24.251273+0000","last_fullsized":"2023-10-06T04:32:24.251273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270989+0000","last_change":"2023-10-06T04:32:24.265155+0000","last_active":"2023-10-06T04:32:24.270989+0000","last_peered":"2023-10-06T04:32:24.270989+0000","last_clean":"2023-10-06T04:32:24.270989+0000","last_became_active":"2023-10-06T04:32:24.264863+0000","last_became_peered":"2023-10-06T04:32:24.264863+0000","last_unstale":"2023-10-06T04:32:24.270989+0000","last_undegraded":"2023-10-06T04:32:24.270989+0000","last_fullsized":"2023-10-06T04:32:24.270989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251056+0000","last_change":"2023-10-06T04:32:23.440295+0000","last_active":"2023-10-06T04:32:24.251056+0000","last_peered":"2023-10-06T04:32:24.251056+0000","last_clean":"2023-10-06T04:32:24.251056+0000","last_became_active":"2023-10-06T04:32:23.440039+0000","last_became_peered":"2023-10-06T04:32:23.440039+0000","last_unstale":"2023-10-06T04:32:24.251056+0000","last_undegraded":"2023-10-06T04:32:24.251056+0000","last_fullsized":"2023-10-06T04:32:24.251056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272396+0000","last_change":"2023-10-06T04:32:24.271227+0000","last_active":"2023-10-06T04:32:24.272396+0000","last_peered":"2023-10-06T04:32:24.272396+0000","last_clean":"2023-10-06T04:32:24.272396+0000","last_became_active":"2023-10-06T04:32:24.270984+0000","last_became_peered":"2023-10-06T04:32:24.270984+0000","last_unstale":"2023-10-06T04:32:24.272396+0000","last_undegraded":"2023-10-06T04:32:24.272396+0000","last_fullsized":"2023-10-06T04:32:24.272396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.260712+0000","last_change":"2023-10-06T04:32:23.447791+0000","last_active":"2023-10-06T04:32:24.260712+0000","last_peered":"2023-10-06T04:32:24.260712+0000","last_clean":"2023-10-06T04:32:24.260712+0000","last_became_active":"2023-10-06T04:32:23.447395+0000","last_became_peered":"2023-10-06T04:32:23.447395+0000","last_unstale":"2023-10-06T04:32:24.260712+0000","last_undegraded":"2023-10-06T04:32:24.260712+0000","last_fullsized":"2023-10-06T04:32:24.260712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:18.839316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274487+0000","last_change":"2023-10-06T04:32:24.269565+0000","last_active":"2023-10-06T04:32:24.274487+0000","last_peered":"2023-10-06T04:32:24.274487+0000","last_clean":"2023-10-06T04:32:24.274487+0000","last_became_active":"2023-10-06T04:32:24.269285+0000","last_became_peered":"2023-10-06T04:32:24.269285+0000","last_unstale":"2023-10-06T04:32:24.274487+0000","last_undegraded":"2023-10-06T04:32:24.274487+0000","last_fullsized":"2023-10-06T04:32:24.274487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.261745+0000","last_change":"2023-10-06T04:32:23.444329+0000","last_active":"2023-10-06T04:32:24.261745+0000","last_peered":"2023-10-06T04:32:24.261745+0000","last_clean":"2023-10-06T04:32:24.261745+0000","last_became_active":"2023-10-06T04:32:23.444020+0000","last_became_peered":"2023-10-06T04:32:23.444020+0000","last_unstale":"2023-10-06T04:32:24.261745+0000","last_undegraded":"2023-10-06T04:32:24.261745+0000","last_fullsized":"2023-10-06T04:32:24.261745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.268975+0000","last_change":"2023-10-06T04:32:24.264060+0000","last_active":"2023-10-06T04:32:24.268975+0000","last_peered":"2023-10-06T04:32:24.268975+0000","last_clean":"2023-10-06T04:32:24.268975+0000","last_became_active":"2023-10-06T04:32:24.263825+0000","last_became_peered":"2023-10-06T04:32:24.263825+0000","last_unstale":"2023-10-06T04:32:24.268975+0000","last_undegraded":"2023-10-06T04:32:24.268975+0000","last_fullsized":"2023-10-06T04:32:24.268975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440264+0000","last_change":"2023-10-06T04:32:23.447887+0000","last_active":"2023-10-06T04:32:24.440264+0000","last_peered":"2023-10-06T04:32:24.440264+0000","last_clean":"2023-10-06T04:32:24.440264+0000","last_became_active":"2023-10-06T04:32:23.447650+0000","last_became_peered":"2023-10-06T04:32:23.447650+0000","last_unstale":"2023-10-06T04:32:24.440264+0000","last_undegraded":"2023-10-06T04:32:24.440264+0000","last_fullsized":"2023-10-06T04:32:24.440264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:47.164678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.277552+0000","last_change":"2023-10-06T04:32:24.270050+0000","last_active":"2023-10-06T04:32:24.277552+0000","last_peered":"2023-10-06T04:32:24.277552+0000","last_clean":"2023-10-06T04:32:24.277552+0000","last_became_active":"2023-10-06T04:32:24.269298+0000","last_became_peered":"2023-10-06T04:32:24.269298+0000","last_unstale":"2023-10-06T04:32:24.277552+0000","last_undegraded":"2023-10-06T04:32:24.277552+0000","last_fullsized":"2023-10-06T04:32:24.277552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.447538+0000","last_change":"2023-10-06T04:32:23.447643+0000","last_active":"2023-10-06T04:32:24.447538+0000","last_peered":"2023-10-06T04:32:24.447538+0000","last_clean":"2023-10-06T04:32:24.447538+0000","last_became_active":"2023-10-06T04:32:23.447352+0000","last_became_peered":"2023-10-06T04:32:23.447352+0000","last_unstale":"2023-10-06T04:32:24.447538+0000","last_undegraded":"2023-10-06T04:32:24.447538+0000","last_fullsized":"2023-10-06T04:32:24.447538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274008+0000","last_change":"2023-10-06T04:32:24.267086+0000","last_active":"2023-10-06T04:32:24.274008+0000","last_peered":"2023-10-06T04:32:24.274008+0000","last_clean":"2023-10-06T04:32:24.274008+0000","last_became_active":"2023-10-06T04:32:24.266829+0000","last_became_peered":"2023-10-06T04:32:24.266829+0000","last_unstale":"2023-10-06T04:32:24.274008+0000","last_undegraded":"2023-10-06T04:32:24.274008+0000","last_fullsized":"2023-10-06T04:32:24.274008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440259+0000","last_change":"2023-10-06T04:32:23.443222+0000","last_active":"2023-10-06T04:32:24.440259+0000","last_peered":"2023-10-06T04:32:24.440259+0000","last_clean":"2023-10-06T04:32:24.440259+0000","last_became_active":"2023-10-06T04:32:23.442957+0000","last_became_peered":"2023-10-06T04:32:23.442957+0000","last_unstale":"2023-10-06T04:32:24.440259+0000","last_undegraded":"2023-10-06T04:32:24.440259+0000","last_fullsized":"2023-10-06T04:32:24.440259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270508+0000","last_change":"2023-10-06T04:32:24.268660+0000","last_active":"2023-10-06T04:32:24.270508+0000","last_peered":"2023-10-06T04:32:24.270508+0000","last_clean":"2023-10-06T04:32:24.270508+0000","last_became_active":"2023-10-06T04:32:24.268397+0000","last_became_peered":"2023-10-06T04:32:24.268397+0000","last_unstale":"2023-10-06T04:32:24.270508+0000","last_undegraded":"2023-10-06T04:32:24.270508+0000","last_fullsized":"2023-10-06T04:32:24.270508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251966+0000","last_change":"2023-10-06T04:32:23.445203+0000","last_active":"2023-10-06T04:32:24.251966+0000","last_peered":"2023-10-06T04:32:24.251966+0000","last_clean":"2023-10-06T04:32:24.251966+0000","last_became_active":"2023-10-06T04:32:23.444688+0000","last_became_peered":"2023-10-06T04:32:23.444688+0000","last_unstale":"2023-10-06T04:32:24.251966+0000","last_undegraded":"2023-10-06T04:32:24.251966+0000","last_fullsized":"2023-10-06T04:32:24.251966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274578+0000","last_change":"2023-10-06T04:32:24.271729+0000","last_active":"2023-10-06T04:32:24.274578+0000","last_peered":"2023-10-06T04:32:24.274578+0000","last_clean":"2023-10-06T04:32:24.274578+0000","last_became_active":"2023-10-06T04:32:24.271453+0000","last_became_peered":"2023-10-06T04:32:24.271453+0000","last_unstale":"2023-10-06T04:32:24.274578+0000","last_undegraded":"2023-10-06T04:32:24.274578+0000","last_fullsized":"2023-10-06T04:32:24.274578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.259047+0000","last_change":"2023-10-06T04:32:23.440530+0000","last_active":"2023-10-06T04:32:24.259047+0000","last_peered":"2023-10-06T04:32:24.259047+0000","last_clean":"2023-10-06T04:32:24.259047+0000","last_became_active":"2023-10-06T04:32:23.440226+0000","last_became_peered":"2023-10-06T04:32:23.440226+0000","last_unstale":"2023-10-06T04:32:24.259047+0000","last_undegraded":"2023-10-06T04:32:24.259047+0000","last_fullsized":"2023-10-06T04:32:24.259047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.271024+0000","last_change":"2023-10-06T04:32:24.268363+0000","last_active":"2023-10-06T04:32:24.271024+0000","last_peered":"2023-10-06T04:32:24.271024+0000","last_clean":"2023-10-06T04:32:24.271024+0000","last_became_active":"2023-10-06T04:32:24.268116+0000","last_became_peered":"2023-10-06T04:32:24.268116+0000","last_unstale":"2023-10-06T04:32:24.271024+0000","last_undegraded":"2023-10-06T04:32:24.271024+0000","last_fullsized":"2023-10-06T04:32:24.271024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446724+0000","last_change":"2023-10-06T04:32:23.452500+0000","last_active":"2023-10-06T04:32:24.446724+0000","last_peered":"2023-10-06T04:32:24.446724+0000","last_clean":"2023-10-06T04:32:24.446724+0000","last_became_active":"2023-10-06T04:32:23.452236+0000","last_became_peered":"2023-10-06T04:32:23.452236+0000","last_unstale":"2023-10-06T04:32:24.446724+0000","last_undegraded":"2023-10-06T04:32:24.446724+0000","last_fullsized":"2023-10-06T04:32:24.446724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270753+0000","last_change":"2023-10-06T04:32:24.265984+0000","last_active":"2023-10-06T04:32:24.270753+0000","last_peered":"2023-10-06T04:32:24.270753+0000","last_clean":"2023-10-06T04:32:24.270753+0000","last_became_active":"2023-10-06T04:32:24.265707+0000","last_became_peered":"2023-10-06T04:32:24.265707+0000","last_unstale":"2023-10-06T04:32:24.270753+0000","last_undegraded":"2023-10-06T04:32:24.270753+0000","last_fullsized":"2023-10-06T04:32:24.270753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446333+0000","last_change":"2023-10-06T04:32:23.447020+0000","last_active":"2023-10-06T04:32:24.446333+0000","last_peered":"2023-10-06T04:32:24.446333+0000","last_clean":"2023-10-06T04:32:24.446333+0000","last_became_active":"2023-10-06T04:32:23.446786+0000","last_became_peered":"2023-10-06T04:32:23.446786+0000","last_unstale":"2023-10-06T04:32:24.446333+0000","last_undegraded":"2023-10-06T04:32:24.446333+0000","last_fullsized":"2023-10-06T04:32:24.446333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.805674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269631+0000","last_change":"2023-10-06T04:32:24.265722+0000","last_active":"2023-10-06T04:32:24.269631+0000","last_peered":"2023-10-06T04:32:24.269631+0000","last_clean":"2023-10-06T04:32:24.269631+0000","last_became_active":"2023-10-06T04:32:24.265435+0000","last_became_peered":"2023-10-06T04:32:24.265435+0000","last_unstale":"2023-10-06T04:32:24.269631+0000","last_undegraded":"2023-10-06T04:32:24.269631+0000","last_fullsized":"2023-10-06T04:32:24.269631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.261738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.442797+0000","last_change":"2023-10-06T04:32:23.442348+0000","last_active":"2023-10-06T04:32:24.442797+0000","last_peered":"2023-10-06T04:32:24.442797+0000","last_clean":"2023-10-06T04:32:24.442797+0000","last_became_active":"2023-10-06T04:32:23.441833+0000","last_became_peered":"2023-10-06T04:32:23.441833+0000","last_unstale":"2023-10-06T04:32:24.442797+0000","last_undegraded":"2023-10-06T04:32:24.442797+0000","last_fullsized":"2023-10-06T04:32:24.442797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.916159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.259088+0000","last_change":"2023-10-06T04:32:24.254622+0000","last_active":"2023-10-06T04:32:24.259088+0000","last_peered":"2023-10-06T04:32:24.259088+0000","last_clean":"2023-10-06T04:32:24.259088+0000","last_became_active":"2023-10-06T04:32:24.254294+0000","last_became_peered":"2023-10-06T04:32:24.254294+0000","last_unstale":"2023-10-06T04:32:24.259088+0000","last_undegraded":"2023-10-06T04:32:24.259088+0000","last_fullsized":"2023-10-06T04:32:24.259088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.443286+0000","last_change":"2023-10-06T04:32:23.447777+0000","last_active":"2023-10-06T04:32:24.443286+0000","last_peered":"2023-10-06T04:32:24.443286+0000","last_clean":"2023-10-06T04:32:24.443286+0000","last_became_active":"2023-10-06T04:32:23.447526+0000","last_became_peered":"2023-10-06T04:32:23.447526+0000","last_unstale":"2023-10-06T04:32:24.443286+0000","last_undegraded":"2023-10-06T04:32:24.443286+0000","last_fullsized":"2023-10-06T04:32:24.443286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:59.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272351+0000","last_change":"2023-10-06T04:32:24.270632+0000","last_active":"2023-10-06T04:32:24.272351+0000","last_peered":"2023-10-06T04:32:24.272351+0000","last_clean":"2023-10-06T04:32:24.272351+0000","last_became_active":"2023-10-06T04:32:24.270263+0000","last_became_peered":"2023-10-06T04:32:24.270263+0000","last_unstale":"2023-10-06T04:32:24.272351+0000","last_undegraded":"2023-10-06T04:32:24.272351+0000","last_fullsized":"2023-10-06T04:32:24.272351+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:19.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251541+0000","last_change":"2023-10-06T04:32:23.446504+0000","last_active":"2023-10-06T04:32:24.251541+0000","last_peered":"2023-10-06T04:32:24.251541+0000","last_clean":"2023-10-06T04:32:24.251541+0000","last_became_active":"2023-10-06T04:32:23.446221+0000","last_became_peered":"2023-10-06T04:32:23.446221+0000","last_unstale":"2023-10-06T04:32:24.251541+0000","last_undegraded":"2023-10-06T04:32:24.251541+0000","last_fullsized":"2023-10-06T04:32:24.251541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274551+0000","last_change":"2023-10-06T04:32:24.271430+0000","last_active":"2023-10-06T04:32:24.274551+0000","last_peered":"2023-10-06T04:32:24.274551+0000","last_clean":"2023-10-06T04:32:24.274551+0000","last_became_active":"2023-10-06T04:32:24.271180+0000","last_became_peered":"2023-10-06T04:32:24.271180+0000","last_unstale":"2023-10-06T04:32:24.274551+0000","last_undegraded":"2023-10-06T04:32:24.274551+0000","last_fullsized":"2023-10-06T04:32:24.274551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.439980+0000","last_change":"2023-10-06T04:32:23.445911+0000","last_active":"2023-10-06T04:32:24.439980+0000","last_peered":"2023-10-06T04:32:24.439980+0000","last_clean":"2023-10-06T04:32:24.439980+0000","last_became_active":"2023-10-06T04:32:23.445582+0000","last_became_peered":"2023-10-06T04:32:23.445582+0000","last_unstale":"2023-10-06T04:32:24.439980+0000","last_undegraded":"2023-10-06T04:32:24.439980+0000","last_fullsized":"2023-10-06T04:32:24.439980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:35.404671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.267532+0000","last_change":"2023-10-06T04:32:24.259250+0000","last_active":"2023-10-06T04:32:24.267532+0000","last_peered":"2023-10-06T04:32:24.267532+0000","last_clean":"2023-10-06T04:32:24.267532+0000","last_became_active":"2023-10-06T04:32:24.258962+0000","last_became_peered":"2023-10-06T04:32:24.258962+0000","last_unstale":"2023-10-06T04:32:24.267532+0000","last_undegraded":"2023-10-06T04:32:24.267532+0000","last_fullsized":"2023-10-06T04:32:24.267532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257714+0000","last_change":"2023-10-06T04:32:23.445529+0000","last_active":"2023-10-06T04:32:24.257714+0000","last_peered":"2023-10-06T04:32:24.257714+0000","last_clean":"2023-10-06T04:32:24.257714+0000","last_became_active":"2023-10-06T04:32:23.445245+0000","last_became_peered":"2023-10-06T04:32:23.445245+0000","last_unstale":"2023-10-06T04:32:24.257714+0000","last_undegraded":"2023-10-06T04:32:24.257714+0000","last_fullsized":"2023-10-06T04:32:24.257714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.277300+0000","last_change":"2023-10-06T04:32:24.270658+0000","last_active":"2023-10-06T04:32:24.277300+0000","last_peered":"2023-10-06T04:32:24.277300+0000","last_clean":"2023-10-06T04:32:24.277300+0000","last_became_active":"2023-10-06T04:32:24.270394+0000","last_became_peered":"2023-10-06T04:32:24.270394+0000","last_unstale":"2023-10-06T04:32:24.277300+0000","last_undegraded":"2023-10-06T04:32:24.277300+0000","last_fullsized":"2023-10-06T04:32:24.277300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.256385+0000","last_change":"2023-10-06T04:32:23.452178+0000","last_active":"2023-10-06T04:32:24.256385+0000","last_peered":"2023-10-06T04:32:24.256385+0000","last_clean":"2023-10-06T04:32:24.256385+0000","last_became_active":"2023-10-06T04:32:23.451849+0000","last_became_peered":"2023-10-06T04:32:23.451849+0000","last_unstale":"2023-10-06T04:32:24.256385+0000","last_undegraded":"2023-10-06T04:32:24.256385+0000","last_fullsized":"2023-10-06T04:32:24.256385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.026875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.271008+0000","last_change":"2023-10-06T04:32:24.265320+0000","last_active":"2023-10-06T04:32:24.271008+0000","last_peered":"2023-10-06T04:32:24.271008+0000","last_clean":"2023-10-06T04:32:24.271008+0000","last_became_active":"2023-10-06T04:32:24.264894+0000","last_became_peered":"2023-10-06T04:32:24.264894+0000","last_unstale":"2023-10-06T04:32:24.271008+0000","last_undegraded":"2023-10-06T04:32:24.271008+0000","last_fullsized":"2023-10-06T04:32:24.271008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.258192+0000","last_change":"2023-10-06T04:32:23.444136+0000","last_active":"2023-10-06T04:32:24.258192+0000","last_peered":"2023-10-06T04:32:24.258192+0000","last_clean":"2023-10-06T04:32:24.258192+0000","last_became_active":"2023-10-06T04:32:23.443839+0000","last_became_peered":"2023-10-06T04:32:23.443839+0000","last_unstale":"2023-10-06T04:32:24.258192+0000","last_undegraded":"2023-10-06T04:32:24.258192+0000","last_fullsized":"2023-10-06T04:32:24.258192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:10.814479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.276784+0000","last_change":"2023-10-06T04:32:24.269672+0000","last_active":"2023-10-06T04:32:24.276784+0000","last_peered":"2023-10-06T04:32:24.276784+0000","last_clean":"2023-10-06T04:32:24.276784+0000","last_became_active":"2023-10-06T04:32:24.269407+0000","last_became_peered":"2023-10-06T04:32:24.269407+0000","last_unstale":"2023-10-06T04:32:24.276784+0000","last_undegraded":"2023-10-06T04:32:24.276784+0000","last_fullsized":"2023-10-06T04:32:24.276784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.252318+0000","last_change":"2023-10-06T04:32:23.446685+0000","last_active":"2023-10-06T04:32:24.252318+0000","last_peered":"2023-10-06T04:32:24.252318+0000","last_clean":"2023-10-06T04:32:24.252318+0000","last_became_active":"2023-10-06T04:32:23.446432+0000","last_became_peered":"2023-10-06T04:32:23.446432+0000","last_unstale":"2023-10-06T04:32:24.252318+0000","last_undegraded":"2023-10-06T04:32:24.252318+0000","last_fullsized":"2023-10-06T04:32:24.252318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251619+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:32:24.251619+0000","last_peered":"2023-10-06T04:32:24.251619+0000","last_clean":"2023-10-06T04:32:24.251619+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:32:24.251619+0000","last_undegraded":"2023-10-06T04:32:24.251619+0000","last_fullsized":"2023-10-06T04:32:24.251619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270201+0000","last_change":"2023-10-06T04:32:24.266084+0000","last_active":"2023-10-06T04:32:24.270201+0000","last_peered":"2023-10-06T04:32:24.270201+0000","last_clean":"2023-10-06T04:32:24.270201+0000","last_became_active":"2023-10-06T04:32:24.265847+0000","last_became_peered":"2023-10-06T04:32:24.265847+0000","last_unstale":"2023-10-06T04:32:24.270201+0000","last_undegraded":"2023-10-06T04:32:24.270201+0000","last_fullsized":"2023-10-06T04:32:24.270201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.252731+0000","last_change":"2023-10-06T04:32:23.451106+0000","last_active":"2023-10-06T04:32:24.252731+0000","last_peered":"2023-10-06T04:32:24.252731+0000","last_clean":"2023-10-06T04:32:24.252731+0000","last_became_active":"2023-10-06T04:32:23.450838+0000","last_became_peered":"2023-10-06T04:32:23.450838+0000","last_unstale":"2023-10-06T04:32:24.252731+0000","last_undegraded":"2023-10-06T04:32:24.252731+0000","last_fullsized":"2023-10-06T04:32:24.252731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.261318+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:32:24.261318+0000","last_peered":"2023-10-06T04:32:24.261318+0000","last_clean":"2023-10-06T04:32:24.261318+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:32:24.261318+0000","last_undegraded":"2023-10-06T04:32:24.261318+0000","last_fullsized":"2023-10-06T04:32:24.261318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274657+0000","last_change":"2023-10-06T04:32:24.272289+0000","last_active":"2023-10-06T04:32:24.274657+0000","last_peered":"2023-10-06T04:32:24.274657+0000","last_clean":"2023-10-06T04:32:24.274657+0000","last_became_active":"2023-10-06T04:32:24.271996+0000","last_became_peered":"2023-10-06T04:32:24.271996+0000","last_unstale":"2023-10-06T04:32:24.274657+0000","last_undegraded":"2023-10-06T04:32:24.274657+0000","last_fullsized":"2023-10-06T04:32:24.274657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.256443+0000","last_change":"2023-10-06T04:32:23.444286+0000","last_active":"2023-10-06T04:32:24.256443+0000","last_peered":"2023-10-06T04:32:24.256443+0000","last_clean":"2023-10-06T04:32:24.256443+0000","last_became_active":"2023-10-06T04:32:23.443982+0000","last_became_peered":"2023-10-06T04:32:23.443982+0000","last_unstale":"2023-10-06T04:32:24.256443+0000","last_undegraded":"2023-10-06T04:32:24.256443+0000","last_fullsized":"2023-10-06T04:32:24.256443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:20.394340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257440+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:32:24.257440+0000","last_peered":"2023-10-06T04:32:24.257440+0000","last_clean":"2023-10-06T04:32:24.257440+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:32:24.257440+0000","last_undegraded":"2023-10-06T04:32:24.257440+0000","last_fullsized":"2023-10-06T04:32:24.257440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269441+0000","last_change":"2023-10-06T04:32:24.265180+0000","last_active":"2023-10-06T04:32:24.269441+0000","last_peered":"2023-10-06T04:32:24.269441+0000","last_clean":"2023-10-06T04:32:24.269441+0000","last_became_active":"2023-10-06T04:32:24.264769+0000","last_became_peered":"2023-10-06T04:32:24.264769+0000","last_unstale":"2023-10-06T04:32:24.269441+0000","last_undegraded":"2023-10-06T04:32:24.269441+0000","last_fullsized":"2023-10-06T04:32:24.269441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257145+0000","last_change":"2023-10-06T04:32:23.452281+0000","last_active":"2023-10-06T04:32:24.257145+0000","last_peered":"2023-10-06T04:32:24.257145+0000","last_clean":"2023-10-06T04:32:24.257145+0000","last_became_active":"2023-10-06T04:32:23.452023+0000","last_became_peered":"2023-10-06T04:32:23.452023+0000","last_unstale":"2023-10-06T04:32:24.257145+0000","last_undegraded":"2023-10-06T04:32:24.257145+0000","last_fullsized":"2023-10-06T04:32:24.257145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.261356+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:32:24.261356+0000","last_peered":"2023-10-06T04:32:24.261356+0000","last_clean":"2023-10-06T04:32:24.261356+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:32:24.261356+0000","last_undegraded":"2023-10-06T04:32:24.261356+0000","last_fullsized":"2023-10-06T04:32:24.261356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257561+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:32:24.257561+0000","last_peered":"2023-10-06T04:32:24.257561+0000","last_clean":"2023-10-06T04:32:24.257561+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:32:24.257561+0000","last_undegraded":"2023-10-06T04:32:24.257561+0000","last_fullsized":"2023-10-06T04:32:24.257561+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270305+0000","last_change":"2023-10-06T04:32:24.267047+0000","last_active":"2023-10-06T04:32:24.270305+0000","last_peered":"2023-10-06T04:32:24.270305+0000","last_clean":"2023-10-06T04:32:24.270305+0000","last_became_active":"2023-10-06T04:32:24.266777+0000","last_became_peered":"2023-10-06T04:32:24.266777+0000","last_unstale":"2023-10-06T04:32:24.270305+0000","last_undegraded":"2023-10-06T04:32:24.270305+0000","last_fullsized":"2023-10-06T04:32:24.270305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440198+0000","last_change":"2023-10-06T04:32:23.448480+0000","last_active":"2023-10-06T04:32:24.440198+0000","last_peered":"2023-10-06T04:32:24.440198+0000","last_clean":"2023-10-06T04:32:24.440198+0000","last_became_active":"2023-10-06T04:32:23.448242+0000","last_became_peered":"2023-10-06T04:32:23.448242+0000","last_unstale":"2023-10-06T04:32:24.440198+0000","last_undegraded":"2023-10-06T04:32:24.440198+0000","last_fullsized":"2023-10-06T04:32:24.440198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257653+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:32:24.257653+0000","last_peered":"2023-10-06T04:32:24.257653+0000","last_clean":"2023-10-06T04:32:24.257653+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:32:24.257653+0000","last_undegraded":"2023-10-06T04:32:24.257653+0000","last_fullsized":"2023-10-06T04:32:24.257653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270236+0000","last_change":"2023-10-06T04:32:24.266050+0000","last_active":"2023-10-06T04:32:24.270236+0000","last_peered":"2023-10-06T04:32:24.270236+0000","last_clean":"2023-10-06T04:32:24.270236+0000","last_became_active":"2023-10-06T04:32:24.265769+0000","last_became_peered":"2023-10-06T04:32:24.265769+0000","last_unstale":"2023-10-06T04:32:24.270236+0000","last_undegraded":"2023-10-06T04:32:24.270236+0000","last_fullsized":"2023-10-06T04:32:24.270236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:01.427293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.263759+0000","last_change":"2023-10-06T04:32:23.443114+0000","last_active":"2023-10-06T04:32:24.263759+0000","last_peered":"2023-10-06T04:32:24.263759+0000","last_clean":"2023-10-06T04:32:24.263759+0000","last_became_active":"2023-10-06T04:32:23.442844+0000","last_became_peered":"2023-10-06T04:32:23.442844+0000","last_unstale":"2023-10-06T04:32:24.263759+0000","last_undegraded":"2023-10-06T04:32:24.263759+0000","last_fullsized":"2023-10-06T04:32:24.263759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257458+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:32:24.257458+0000","last_peered":"2023-10-06T04:32:24.257458+0000","last_clean":"2023-10-06T04:32:24.257458+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:32:24.257458+0000","last_undegraded":"2023-10-06T04:32:24.257458+0000","last_fullsized":"2023-10-06T04:32:24.257458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274623+0000","last_change":"2023-10-06T04:32:24.265505+0000","last_active":"2023-10-06T04:32:24.274623+0000","last_peered":"2023-10-06T04:32:24.274623+0000","last_clean":"2023-10-06T04:32:24.274623+0000","last_became_active":"2023-10-06T04:32:24.265228+0000","last_became_peered":"2023-10-06T04:32:24.265228+0000","last_unstale":"2023-10-06T04:32:24.274623+0000","last_undegraded":"2023-10-06T04:32:24.274623+0000","last_fullsized":"2023-10-06T04:32:24.274623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.250899+0000","last_change":"2023-10-06T04:32:23.438229+0000","last_active":"2023-10-06T04:32:24.250899+0000","last_peered":"2023-10-06T04:32:24.250899+0000","last_clean":"2023-10-06T04:32:24.250899+0000","last_became_active":"2023-10-06T04:32:23.437954+0000","last_became_peered":"2023-10-06T04:32:23.437954+0000","last_unstale":"2023-10-06T04:32:24.250899+0000","last_undegraded":"2023-10-06T04:32:24.250899+0000","last_fullsized":"2023-10-06T04:32:24.250899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:22.972008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251970+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:32:24.251970+0000","last_peered":"2023-10-06T04:32:24.251970+0000","last_clean":"2023-10-06T04:32:24.251970+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:32:24.251970+0000","last_undegraded":"2023-10-06T04:32:24.251970+0000","last_fullsized":"2023-10-06T04:32:24.251970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274648+0000","last_change":"2023-10-06T04:32:24.272223+0000","last_active":"2023-10-06T04:32:24.274648+0000","last_peered":"2023-10-06T04:32:24.274648+0000","last_clean":"2023-10-06T04:32:24.274648+0000","last_became_active":"2023-10-06T04:32:24.271905+0000","last_became_peered":"2023-10-06T04:32:24.271905+0000","last_unstale":"2023-10-06T04:32:24.274648+0000","last_undegraded":"2023-10-06T04:32:24.274648+0000","last_fullsized":"2023-10-06T04:32:24.274648+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:17.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.260527+0000","last_change":"2023-10-06T04:32:23.444321+0000","last_active":"2023-10-06T04:32:24.260527+0000","last_peered":"2023-10-06T04:32:24.260527+0000","last_clean":"2023-10-06T04:32:24.260527+0000","last_became_active":"2023-10-06T04:32:23.444073+0000","last_became_peered":"2023-10-06T04:32:23.444073+0000","last_unstale":"2023-10-06T04:32:24.260527+0000","last_undegraded":"2023-10-06T04:32:24.260527+0000","last_fullsized":"2023-10-06T04:32:24.260527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:37.675115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.262534+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:32:24.262534+0000","last_peered":"2023-10-06T04:32:24.262534+0000","last_clean":"2023-10-06T04:32:24.262534+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:32:24.262534+0000","last_undegraded":"2023-10-06T04:32:24.262534+0000","last_fullsized":"2023-10-06T04:32:24.262534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":983040,"data_stored":741769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.501}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":150349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.746},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.495}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40699999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.32500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.434},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.753}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":740197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.32300000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.434},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.41099999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":158006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.24399999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.497}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":150371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.437},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.627},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.63900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":149907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.375},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.503},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:32:28.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:32:28.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.654+0000 7f19f9f12700 1 Processor -- start 2023-10-06T04:32:28.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.654+0000 7f19f9f12700 1 -- start start 2023-10-06T04:32:28.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.654+0000 7f19f9f12700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f19f4068c90 0x7f19f4156de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.654+0000 7f19f9f12700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 0x7f19f40c3840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.654+0000 7f19f9f12700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19f40cc190 con 0x7f19f40688c0 2023-10-06T04:32:28.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.654+0000 7f19f9f12700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19f40cc620 con 0x7f19f4068c90 2023-10-06T04:32:28.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f9f12700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cb370 con 0x7f19f4157320 2023-10-06T04:32:28.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f37fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f19f40688c0 0x7f19f406a3f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40410/0 (socket says 172.21.15.123:40410) 2023-10-06T04:32:28.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f37fe700 1 -- 172.21.15.123:0/3563780630 learned_addr learned my addr 172.21.15.123:0/3563780630 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:28.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f3fff700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 0x7f19f40c3840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f2ffd700 1 --2- 172.21.15.123:0/3563780630 >> v2:172.21.15.177:3300/0 conn(0x7f19f4068c90 0x7f19f4156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519872145 0 0) 0x7f19f40cc190 con 0x7f19f40688c0 2023-10-06T04:32:28.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f19e0003570 con 0x7f19f40688c0 2023-10-06T04:32:28.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 821154297 0 0) 0x7f19e0003570 con 0x7f19f40688c0 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 msgr2=0x7f19f40c3840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 0x7f19f40c3840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 >> v2:172.21.15.177:3300/0 conn(0x7f19f4068c90 msgr2=0x7f19f4156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 --2- 172.21.15.123:0/3563780630 >> v2:172.21.15.177:3300/0 conn(0x7f19f4068c90 0x7f19f4156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f406ab00 con 0x7f19f40688c0 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f19dc001470 con 0x7f19f40688c0 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 >> v1:172.21.15.123:6789/0 conn(0x7f19f40688c0 legacy=0x7f19f406a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 0x7f19e0003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e0003fe0 0x7f19e0004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e0004d50 0x7f19e00051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cc620 con 0x7f19e0003fe0 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40cb370 con 0x7f19e0004d50 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19e000ba90 con 0x7f19f4157320 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f2ffd700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e0003fe0 0x7f19e0004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f37fe700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 0x7f19e0003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.655+0000 7f19f3fff700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e0004d50 0x7f19e00051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f2ffd700 1 -- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 msgr2=0x7f19e0003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:28.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f2ffd700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19f4157320 0x7f19e0003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f2ffd700 1 -- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e0004d50 msgr2=0x7f19e00051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:28.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f2ffd700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e0004d50 0x7f19e00051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f2ffd700 1 -- 172.21.15.123:0/3563780630 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19e0003570 con 0x7f19e0003fe0 2023-10-06T04:32:28.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f3fff700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19e0004d50 0x7f19e00051c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:28.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f2ffd700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e0003fe0 0x7f19e0004810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009c00 tx=0x7f19e40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca00ed2bdda5726a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400e6b0 con 0x7f19e0003fe0 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19e400e810 con 0x7f19e0003fe0 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f27fc700 1 -- 172.21.15.123:0/3563780630 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19e400ebd0 con 0x7f19e0003fe0 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f9f12700 1 -- 172.21.15.123:0/3563780630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e0003fe0 msgr2=0x7f19e0004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f9f12700 1 --2- 172.21.15.123:0/3563780630 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19e0003fe0 0x7f19e0004810 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f19e4009c00 tx=0x7f19e40091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f3fff700 1 -- 172.21.15.123:0/3563780630 reap_dead start 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f9f12700 1 -- 172.21.15.123:0/3563780630 shutdown_connections 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f9f12700 1 -- 172.21.15.123:0/3563780630 >> 172.21.15.123:0/3563780630 conn(0x7f19f40c9c90 msgr2=0x7f19f406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f9f12700 1 -- 172.21.15.123:0/3563780630 shutdown_connections 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.656+0000 7f19f9f12700 1 -- 172.21.15.123:0/3563780630 wait complete. 2023-10-06T04:32:28.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.658+0000 7f19f9f12700 1 Processor -- start 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.658+0000 7f19f9f12700 1 -- start start 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f9f12700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e0003fe0 0x7f19f4169740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f9f12700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19f40c89e0 0x7f19f40c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f9f12700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f416c490 0x7f19f40ca060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f9f12700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40c7310 con 0x7f19f416c490 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f9f12700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19f41590f0 con 0x7f19f40c89e0 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f9f12700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19f40c8e50 con 0x7f19e0003fe0 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f416c490 0x7f19f40ca060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f416c490 0x7f19f40ca060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35872/0 (socket says 172.21.15.123:35872) 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 -- 172.21.15.123:0/303111323 learned_addr learned my addr 172.21.15.123:0/303111323 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f37fe700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e0003fe0 0x7f19f4169740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f2ffd700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19f40c89e0 0x7f19f40c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 -- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e0003fe0 msgr2=0x7f19f4169740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e0003fe0 0x7f19f4169740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 -- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19f40c89e0 msgr2=0x7f19f40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19f40c89e0 0x7f19f40c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.659+0000 7f19f3fff700 1 -- 172.21.15.123:0/303111323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19f4159280 con 0x7f19f416c490 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.660+0000 7f19f3fff700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f416c490 0x7f19f40ca060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f19f415af00 tx=0x7f19ec00b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.660+0000 7f19eb7fe700 1 -- 172.21.15.123:0/303111323 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec01b070 con 0x7f19f416c490 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.660+0000 7f19eb7fe700 1 -- 172.21.15.123:0/303111323 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19ec00f040 con 0x7f19f416c490 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.660+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19f40c7500 con 0x7f19f416c490 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.660+0000 7f19eb7fe700 1 -- 172.21.15.123:0/303111323 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19ec00e050 con 0x7f19f416c490 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.660+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19f40c7ac0 con 0x7f19f416c490 2023-10-06T04:32:28.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.661+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19f4159410 con 0x7f19f416c490 2023-10-06T04:32:28.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.661+0000 7f19eb7fe700 1 -- 172.21.15.123:0/303111323 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f19ec003020 con 0x7f19f416c490 2023-10-06T04:32:28.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.662+0000 7f19eb7fe700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f19d4066f30 0x7f19d40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:28.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.662+0000 7f19eb7fe700 1 -- 172.21.15.123:0/303111323 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f19ec084960 con 0x7f19f416c490 2023-10-06T04:32:28.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.662+0000 7f19f37fe700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f19d4066f30 0x7f19d40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:28.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.662+0000 7f19f37fe700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f19d4066f30 0x7f19d40693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19dc001620 tx=0x7f19dc001e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:28.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.666+0000 7f19eb7fe700 1 -- 172.21.15.123:0/303111323 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f19ec051a70 con 0x7f19f416c490 2023-10-06T04:32:28.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.844+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f19f40cb580 con 0x7f19d4066f30 2023-10-06T04:32:28.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.852+0000 7f19eb7fe700 1 -- 172.21.15.123:0/303111323 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+311185 (secure 0 0 0) 0x7f19f40cb580 con 0x7f19d4066f30 2023-10-06T04:32:28.858 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f19d4066f30 msgr2=0x7f19d40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f19d4066f30 0x7f19d40693e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f19dc001620 tx=0x7f19dc001e90 comp rx=0 tx=0).stop 2023-10-06T04:32:28.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f416c490 msgr2=0x7f19f40ca060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f416c490 0x7f19f40ca060 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f19f415af00 tx=0x7f19ec00b040 comp rx=0 tx=0).stop 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 shutdown_connections 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f19d4066f30 0x7f19d40693e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f19f416c490 0x7f19f40ca060 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f19e0003fe0 0x7f19f4169740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 --2- 172.21.15.123:0/303111323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f19f40c89e0 0x7f19f40c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.855+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 >> 172.21.15.123:0/303111323 conn(0x7f19f40c9c90 msgr2=0x7f19f40c38b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.856+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 shutdown_connections 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:28.856+0000 7f19f9f12700 1 -- 172.21.15.123:0/303111323 wait complete. 2023-10-06T04:32:28.861 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:32:28.870 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":60,"stamp":"2023-10-06T04:32:26.899805+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592674,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2429,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":147,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2382452,"kb_used_data":4148,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752592268,"statfs":{"total":773094113280,"available":770654482432,"internally_reserved":0,"allocated":4247552,"data_stored":2390349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.476096"},"pg_stats":[{"pgid":"6.3f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270484+0000","last_change":"2023-10-06T04:32:24.268479+0000","last_active":"2023-10-06T04:32:24.270484+0000","last_peered":"2023-10-06T04:32:24.270484+0000","last_clean":"2023-10-06T04:32:24.270484+0000","last_became_active":"2023-10-06T04:32:24.268175+0000","last_became_peered":"2023-10-06T04:32:24.268175+0000","last_unstale":"2023-10-06T04:32:24.270484+0000","last_undegraded":"2023-10-06T04:32:24.270484+0000","last_fullsized":"2023-10-06T04:32:24.270484+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272058+0000","last_change":"2023-10-06T04:32:24.269331+0000","last_active":"2023-10-06T04:32:24.272058+0000","last_peered":"2023-10-06T04:32:24.272058+0000","last_clean":"2023-10-06T04:32:24.272058+0000","last_became_active":"2023-10-06T04:32:24.269096+0000","last_became_peered":"2023-10-06T04:32:24.269096+0000","last_unstale":"2023-10-06T04:32:24.272058+0000","last_undegraded":"2023-10-06T04:32:24.272058+0000","last_fullsized":"2023-10-06T04:32:24.272058+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274016+0000","last_change":"2023-10-06T04:32:24.267252+0000","last_active":"2023-10-06T04:32:24.274016+0000","last_peered":"2023-10-06T04:32:24.274016+0000","last_clean":"2023-10-06T04:32:24.274016+0000","last_became_active":"2023-10-06T04:32:24.266928+0000","last_became_peered":"2023-10-06T04:32:24.266928+0000","last_unstale":"2023-10-06T04:32:24.274016+0000","last_undegraded":"2023-10-06T04:32:24.274016+0000","last_fullsized":"2023-10-06T04:32:24.274016+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272415+0000","last_change":"2023-10-06T04:32:24.271555+0000","last_active":"2023-10-06T04:32:24.272415+0000","last_peered":"2023-10-06T04:32:24.272415+0000","last_clean":"2023-10-06T04:32:24.272415+0000","last_became_active":"2023-10-06T04:32:24.271271+0000","last_became_peered":"2023-10-06T04:32:24.271271+0000","last_unstale":"2023-10-06T04:32:24.272415+0000","last_undegraded":"2023-10-06T04:32:24.272415+0000","last_fullsized":"2023-10-06T04:32:24.272415+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.271957+0000","last_change":"2023-10-06T04:32:24.270442+0000","last_active":"2023-10-06T04:32:24.271957+0000","last_peered":"2023-10-06T04:32:24.271957+0000","last_clean":"2023-10-06T04:32:24.271957+0000","last_became_active":"2023-10-06T04:32:24.270131+0000","last_became_peered":"2023-10-06T04:32:24.270131+0000","last_unstale":"2023-10-06T04:32:24.271957+0000","last_undegraded":"2023-10-06T04:32:24.271957+0000","last_fullsized":"2023-10-06T04:32:24.271957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269930+0000","last_change":"2023-10-06T04:32:24.263272+0000","last_active":"2023-10-06T04:32:24.269930+0000","last_peered":"2023-10-06T04:32:24.269930+0000","last_clean":"2023-10-06T04:32:24.269930+0000","last_became_active":"2023-10-06T04:32:24.262984+0000","last_became_peered":"2023-10-06T04:32:24.262984+0000","last_unstale":"2023-10-06T04:32:24.269930+0000","last_undegraded":"2023-10-06T04:32:24.269930+0000","last_fullsized":"2023-10-06T04:32:24.269930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:48.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270902+0000","last_change":"2023-10-06T04:32:24.263913+0000","last_active":"2023-10-06T04:32:24.270902+0000","last_peered":"2023-10-06T04:32:24.270902+0000","last_clean":"2023-10-06T04:32:24.270902+0000","last_became_active":"2023-10-06T04:32:24.263358+0000","last_became_peered":"2023-10-06T04:32:24.263358+0000","last_unstale":"2023-10-06T04:32:24.270902+0000","last_undegraded":"2023-10-06T04:32:24.270902+0000","last_fullsized":"2023-10-06T04:32:24.270902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270266+0000","last_change":"2023-10-06T04:32:24.266865+0000","last_active":"2023-10-06T04:32:24.270266+0000","last_peered":"2023-10-06T04:32:24.270266+0000","last_clean":"2023-10-06T04:32:24.270266+0000","last_became_active":"2023-10-06T04:32:24.266456+0000","last_became_peered":"2023-10-06T04:32:24.266456+0000","last_unstale":"2023-10-06T04:32:24.270266+0000","last_undegraded":"2023-10-06T04:32:24.270266+0000","last_fullsized":"2023-10-06T04:32:24.270266+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274060+0000","last_change":"2023-10-06T04:32:24.267567+0000","last_active":"2023-10-06T04:32:24.274060+0000","last_peered":"2023-10-06T04:32:24.274060+0000","last_clean":"2023-10-06T04:32:24.274060+0000","last_became_active":"2023-10-06T04:32:24.267326+0000","last_became_peered":"2023-10-06T04:32:24.267326+0000","last_unstale":"2023-10-06T04:32:24.274060+0000","last_undegraded":"2023-10-06T04:32:24.274060+0000","last_fullsized":"2023-10-06T04:32:24.274060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269694+0000","last_change":"2023-10-06T04:32:24.266439+0000","last_active":"2023-10-06T04:32:24.269694+0000","last_peered":"2023-10-06T04:32:24.269694+0000","last_clean":"2023-10-06T04:32:24.269694+0000","last_became_active":"2023-10-06T04:32:24.266172+0000","last_became_peered":"2023-10-06T04:32:24.266172+0000","last_unstale":"2023-10-06T04:32:24.269694+0000","last_undegraded":"2023-10-06T04:32:24.269694+0000","last_fullsized":"2023-10-06T04:32:24.269694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:07.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269612+0000","last_change":"2023-10-06T04:32:24.264976+0000","last_active":"2023-10-06T04:32:24.269612+0000","last_peered":"2023-10-06T04:32:24.269612+0000","last_clean":"2023-10-06T04:32:24.269612+0000","last_became_active":"2023-10-06T04:32:24.264647+0000","last_became_peered":"2023-10-06T04:32:24.264647+0000","last_unstale":"2023-10-06T04:32:24.269612+0000","last_undegraded":"2023-10-06T04:32:24.269612+0000","last_fullsized":"2023-10-06T04:32:24.269612+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.268980+0000","last_change":"2023-10-06T04:32:24.262121+0000","last_active":"2023-10-06T04:32:24.268980+0000","last_peered":"2023-10-06T04:32:24.268980+0000","last_clean":"2023-10-06T04:32:24.268980+0000","last_became_active":"2023-10-06T04:32:24.261846+0000","last_became_peered":"2023-10-06T04:32:24.261846+0000","last_unstale":"2023-10-06T04:32:24.268980+0000","last_undegraded":"2023-10-06T04:32:24.268980+0000","last_fullsized":"2023-10-06T04:32:24.268980+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270791+0000","last_change":"2023-10-06T04:32:24.267116+0000","last_active":"2023-10-06T04:32:24.270791+0000","last_peered":"2023-10-06T04:32:24.270791+0000","last_clean":"2023-10-06T04:32:24.270791+0000","last_became_active":"2023-10-06T04:32:24.266756+0000","last_became_peered":"2023-10-06T04:32:24.266756+0000","last_unstale":"2023-10-06T04:32:24.270791+0000","last_undegraded":"2023-10-06T04:32:24.270791+0000","last_fullsized":"2023-10-06T04:32:24.270791+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:26.457033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274722+0000","last_change":"2023-10-06T04:32:24.267554+0000","last_active":"2023-10-06T04:32:24.274722+0000","last_peered":"2023-10-06T04:32:24.274722+0000","last_clean":"2023-10-06T04:32:24.274722+0000","last_became_active":"2023-10-06T04:32:24.266198+0000","last_became_peered":"2023-10-06T04:32:24.266198+0000","last_unstale":"2023-10-06T04:32:24.274722+0000","last_undegraded":"2023-10-06T04:32:24.274722+0000","last_fullsized":"2023-10-06T04:32:24.274722+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270938+0000","last_change":"2023-10-06T04:32:24.265234+0000","last_active":"2023-10-06T04:32:24.270938+0000","last_peered":"2023-10-06T04:32:24.270938+0000","last_clean":"2023-10-06T04:32:24.270938+0000","last_became_active":"2023-10-06T04:32:24.264892+0000","last_became_peered":"2023-10-06T04:32:24.264892+0000","last_unstale":"2023-10-06T04:32:24.270938+0000","last_undegraded":"2023-10-06T04:32:24.270938+0000","last_fullsized":"2023-10-06T04:32:24.270938+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272961+0000","last_change":"2023-10-06T04:32:24.268696+0000","last_active":"2023-10-06T04:32:24.272961+0000","last_peered":"2023-10-06T04:32:24.272961+0000","last_clean":"2023-10-06T04:32:24.272961+0000","last_became_active":"2023-10-06T04:32:24.268430+0000","last_became_peered":"2023-10-06T04:32:24.268430+0000","last_unstale":"2023-10-06T04:32:24.272961+0000","last_undegraded":"2023-10-06T04:32:24.272961+0000","last_fullsized":"2023-10-06T04:32:24.272961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272716+0000","last_change":"2023-10-06T04:32:24.269272+0000","last_active":"2023-10-06T04:32:24.272716+0000","last_peered":"2023-10-06T04:32:24.272716+0000","last_clean":"2023-10-06T04:32:24.272716+0000","last_became_active":"2023-10-06T04:32:24.268793+0000","last_became_peered":"2023-10-06T04:32:24.268793+0000","last_unstale":"2023-10-06T04:32:24.272716+0000","last_undegraded":"2023-10-06T04:32:24.272716+0000","last_fullsized":"2023-10-06T04:32:24.272716+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273449+0000","last_change":"2023-10-06T04:32:24.268089+0000","last_active":"2023-10-06T04:32:24.273449+0000","last_peered":"2023-10-06T04:32:24.273449+0000","last_clean":"2023-10-06T04:32:24.273449+0000","last_became_active":"2023-10-06T04:32:24.267842+0000","last_became_peered":"2023-10-06T04:32:24.267842+0000","last_unstale":"2023-10-06T04:32:24.273449+0000","last_undegraded":"2023-10-06T04:32:24.273449+0000","last_fullsized":"2023-10-06T04:32:24.273449+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:36.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269498+0000","last_change":"2023-10-06T04:32:24.263185+0000","last_active":"2023-10-06T04:32:24.269498+0000","last_peered":"2023-10-06T04:32:24.269498+0000","last_clean":"2023-10-06T04:32:24.269498+0000","last_became_active":"2023-10-06T04:32:24.262500+0000","last_became_peered":"2023-10-06T04:32:24.262500+0000","last_unstale":"2023-10-06T04:32:24.269498+0000","last_undegraded":"2023-10-06T04:32:24.269498+0000","last_fullsized":"2023-10-06T04:32:24.269498+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:18.096304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272902+0000","last_change":"2023-10-06T04:32:24.266364+0000","last_active":"2023-10-06T04:32:24.272902+0000","last_peered":"2023-10-06T04:32:24.272902+0000","last_clean":"2023-10-06T04:32:24.272902+0000","last_became_active":"2023-10-06T04:32:24.266082+0000","last_became_peered":"2023-10-06T04:32:24.266082+0000","last_unstale":"2023-10-06T04:32:24.272902+0000","last_undegraded":"2023-10-06T04:32:24.272902+0000","last_fullsized":"2023-10-06T04:32:24.272902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269519+0000","last_change":"2023-10-06T04:32:24.265143+0000","last_active":"2023-10-06T04:32:24.269519+0000","last_peered":"2023-10-06T04:32:24.269519+0000","last_clean":"2023-10-06T04:32:24.269519+0000","last_became_active":"2023-10-06T04:32:24.264481+0000","last_became_peered":"2023-10-06T04:32:24.264481+0000","last_unstale":"2023-10-06T04:32:24.269519+0000","last_undegraded":"2023-10-06T04:32:24.269519+0000","last_fullsized":"2023-10-06T04:32:24.269519+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:04.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269525+0000","last_change":"2023-10-06T04:32:24.262815+0000","last_active":"2023-10-06T04:32:24.269525+0000","last_peered":"2023-10-06T04:32:24.269525+0000","last_clean":"2023-10-06T04:32:24.269525+0000","last_became_active":"2023-10-06T04:32:24.262576+0000","last_became_peered":"2023-10-06T04:32:24.262576+0000","last_unstale":"2023-10-06T04:32:24.269525+0000","last_undegraded":"2023-10-06T04:32:24.269525+0000","last_fullsized":"2023-10-06T04:32:24.269525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272837+0000","last_change":"2023-10-06T04:32:24.269201+0000","last_active":"2023-10-06T04:32:24.272837+0000","last_peered":"2023-10-06T04:32:24.272837+0000","last_clean":"2023-10-06T04:32:24.272837+0000","last_became_active":"2023-10-06T04:32:24.268841+0000","last_became_peered":"2023-10-06T04:32:24.268841+0000","last_unstale":"2023-10-06T04:32:24.272837+0000","last_undegraded":"2023-10-06T04:32:24.272837+0000","last_fullsized":"2023-10-06T04:32:24.272837+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270172+0000","last_change":"2023-10-06T04:32:24.264875+0000","last_active":"2023-10-06T04:32:24.270172+0000","last_peered":"2023-10-06T04:32:24.270172+0000","last_clean":"2023-10-06T04:32:24.270172+0000","last_became_active":"2023-10-06T04:32:24.264601+0000","last_became_peered":"2023-10-06T04:32:24.264601+0000","last_unstale":"2023-10-06T04:32:24.270172+0000","last_undegraded":"2023-10-06T04:32:24.270172+0000","last_fullsized":"2023-10-06T04:32:24.270172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:38.680988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273472+0000","last_change":"2023-10-06T04:32:24.268714+0000","last_active":"2023-10-06T04:32:24.273472+0000","last_peered":"2023-10-06T04:32:24.273472+0000","last_clean":"2023-10-06T04:32:24.273472+0000","last_became_active":"2023-10-06T04:32:24.268440+0000","last_became_peered":"2023-10-06T04:32:24.268440+0000","last_unstale":"2023-10-06T04:32:24.273472+0000","last_undegraded":"2023-10-06T04:32:24.273472+0000","last_fullsized":"2023-10-06T04:32:24.273472+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:24.913639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270362+0000","last_change":"2023-10-06T04:32:24.266945+0000","last_active":"2023-10-06T04:32:24.270362+0000","last_peered":"2023-10-06T04:32:24.270362+0000","last_clean":"2023-10-06T04:32:24.270362+0000","last_became_active":"2023-10-06T04:32:24.266673+0000","last_became_peered":"2023-10-06T04:32:24.266673+0000","last_unstale":"2023-10-06T04:32:24.270362+0000","last_undegraded":"2023-10-06T04:32:24.270362+0000","last_fullsized":"2023-10-06T04:32:24.270362+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273762+0000","last_change":"2023-10-06T04:32:24.270531+0000","last_active":"2023-10-06T04:32:24.273762+0000","last_peered":"2023-10-06T04:32:24.273762+0000","last_clean":"2023-10-06T04:32:24.273762+0000","last_became_active":"2023-10-06T04:32:24.270268+0000","last_became_peered":"2023-10-06T04:32:24.270268+0000","last_unstale":"2023-10-06T04:32:24.273762+0000","last_undegraded":"2023-10-06T04:32:24.273762+0000","last_fullsized":"2023-10-06T04:32:24.273762+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:34.859497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269547+0000","last_change":"2023-10-06T04:32:24.265317+0000","last_active":"2023-10-06T04:32:24.269547+0000","last_peered":"2023-10-06T04:32:24.269547+0000","last_clean":"2023-10-06T04:32:24.269547+0000","last_became_active":"2023-10-06T04:32:24.264377+0000","last_became_peered":"2023-10-06T04:32:24.264377+0000","last_unstale":"2023-10-06T04:32:24.269547+0000","last_undegraded":"2023-10-06T04:32:24.269547+0000","last_fullsized":"2023-10-06T04:32:24.269547+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272464+0000","last_change":"2023-10-06T04:32:24.264442+0000","last_active":"2023-10-06T04:32:24.272464+0000","last_peered":"2023-10-06T04:32:24.272464+0000","last_clean":"2023-10-06T04:32:24.272464+0000","last_became_active":"2023-10-06T04:32:24.264148+0000","last_became_peered":"2023-10-06T04:32:24.264148+0000","last_unstale":"2023-10-06T04:32:24.272464+0000","last_undegraded":"2023-10-06T04:32:24.272464+0000","last_fullsized":"2023-10-06T04:32:24.272464+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.273792+0000","last_change":"2023-10-06T04:32:24.270800+0000","last_active":"2023-10-06T04:32:24.273792+0000","last_peered":"2023-10-06T04:32:24.273792+0000","last_clean":"2023-10-06T04:32:24.273792+0000","last_became_active":"2023-10-06T04:32:24.270560+0000","last_became_peered":"2023-10-06T04:32:24.270560+0000","last_unstale":"2023-10-06T04:32:24.273792+0000","last_undegraded":"2023-10-06T04:32:24.273792+0000","last_fullsized":"2023-10-06T04:32:24.273792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274625+0000","last_change":"2023-10-06T04:32:24.271609+0000","last_active":"2023-10-06T04:32:24.274625+0000","last_peered":"2023-10-06T04:32:24.274625+0000","last_clean":"2023-10-06T04:32:24.274625+0000","last_became_active":"2023-10-06T04:32:24.271354+0000","last_became_peered":"2023-10-06T04:32:24.271354+0000","last_unstale":"2023-10-06T04:32:24.274625+0000","last_undegraded":"2023-10-06T04:32:24.274625+0000","last_fullsized":"2023-10-06T04:32:24.274625+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270797+0000","last_change":"2023-10-06T04:32:24.263719+0000","last_active":"2023-10-06T04:32:24.270797+0000","last_peered":"2023-10-06T04:32:24.270797+0000","last_clean":"2023-10-06T04:32:24.270797+0000","last_became_active":"2023-10-06T04:32:24.263442+0000","last_became_peered":"2023-10-06T04:32:24.263442+0000","last_unstale":"2023-10-06T04:32:24.270797+0000","last_undegraded":"2023-10-06T04:32:24.270797+0000","last_fullsized":"2023-10-06T04:32:24.270797+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:40.811547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274529+0000","last_change":"2023-10-06T04:32:24.270093+0000","last_active":"2023-10-06T04:32:24.274529+0000","last_peered":"2023-10-06T04:32:24.274529+0000","last_clean":"2023-10-06T04:32:24.274529+0000","last_became_active":"2023-10-06T04:32:24.269851+0000","last_became_peered":"2023-10-06T04:32:24.269851+0000","last_unstale":"2023-10-06T04:32:24.274529+0000","last_undegraded":"2023-10-06T04:32:24.274529+0000","last_fullsized":"2023-10-06T04:32:24.274529+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"33'4","reported_seq":14,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446192+0000","last_change":"2023-10-06T04:32:23.441194+0000","last_active":"2023-10-06T04:32:24.446192+0000","last_peered":"2023-10-06T04:32:24.446192+0000","last_clean":"2023-10-06T04:32:24.446192+0000","last_became_active":"2023-10-06T04:32:23.440849+0000","last_became_peered":"2023-10-06T04:32:23.440849+0000","last_unstale":"2023-10-06T04:32:24.446192+0000","last_undegraded":"2023-10-06T04:32:24.446192+0000","last_fullsized":"2023-10-06T04:32:24.446192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:59.829363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":554,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270962+0000","last_change":"2023-10-06T04:32:24.267279+0000","last_active":"2023-10-06T04:32:24.270962+0000","last_peered":"2023-10-06T04:32:24.270962+0000","last_clean":"2023-10-06T04:32:24.270962+0000","last_became_active":"2023-10-06T04:32:24.267016+0000","last_became_peered":"2023-10-06T04:32:24.267016+0000","last_unstale":"2023-10-06T04:32:24.270962+0000","last_undegraded":"2023-10-06T04:32:24.270962+0000","last_fullsized":"2023-10-06T04:32:24.270962+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.443655+0000","last_change":"2023-10-06T04:32:23.450031+0000","last_active":"2023-10-06T04:32:24.443655+0000","last_peered":"2023-10-06T04:32:24.443655+0000","last_clean":"2023-10-06T04:32:24.443655+0000","last_became_active":"2023-10-06T04:32:23.449717+0000","last_became_peered":"2023-10-06T04:32:23.449717+0000","last_unstale":"2023-10-06T04:32:24.443655+0000","last_undegraded":"2023-10-06T04:32:24.443655+0000","last_fullsized":"2023-10-06T04:32:24.443655+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:19.596388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274702+0000","last_change":"2023-10-06T04:32:24.267302+0000","last_active":"2023-10-06T04:32:24.274702+0000","last_peered":"2023-10-06T04:32:24.274702+0000","last_clean":"2023-10-06T04:32:24.274702+0000","last_became_active":"2023-10-06T04:32:24.266976+0000","last_became_peered":"2023-10-06T04:32:24.266976+0000","last_unstale":"2023-10-06T04:32:24.274702+0000","last_undegraded":"2023-10-06T04:32:24.274702+0000","last_fullsized":"2023-10-06T04:32:24.274702+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.443427+0000","last_change":"2023-10-06T04:32:23.447029+0000","last_active":"2023-10-06T04:32:24.443427+0000","last_peered":"2023-10-06T04:32:24.443427+0000","last_clean":"2023-10-06T04:32:24.443427+0000","last_became_active":"2023-10-06T04:32:23.446659+0000","last_became_peered":"2023-10-06T04:32:23.446659+0000","last_unstale":"2023-10-06T04:32:24.443427+0000","last_undegraded":"2023-10-06T04:32:24.443427+0000","last_fullsized":"2023-10-06T04:32:24.443427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:13.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270165+0000","last_change":"2023-10-06T04:32:24.265927+0000","last_active":"2023-10-06T04:32:24.270165+0000","last_peered":"2023-10-06T04:32:24.270165+0000","last_clean":"2023-10-06T04:32:24.270165+0000","last_became_active":"2023-10-06T04:32:24.265668+0000","last_became_peered":"2023-10-06T04:32:24.265668+0000","last_unstale":"2023-10-06T04:32:24.270165+0000","last_undegraded":"2023-10-06T04:32:24.270165+0000","last_fullsized":"2023-10-06T04:32:24.270165+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.442929+0000","last_change":"2023-10-06T04:32:23.437810+0000","last_active":"2023-10-06T04:32:24.442929+0000","last_peered":"2023-10-06T04:32:24.442929+0000","last_clean":"2023-10-06T04:32:24.442929+0000","last_became_active":"2023-10-06T04:32:23.437533+0000","last_became_peered":"2023-10-06T04:32:23.437533+0000","last_unstale":"2023-10-06T04:32:24.442929+0000","last_undegraded":"2023-10-06T04:32:24.442929+0000","last_fullsized":"2023-10-06T04:32:24.442929+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:46.149228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.268930+0000","last_change":"2023-10-06T04:32:24.261934+0000","last_active":"2023-10-06T04:32:24.268930+0000","last_peered":"2023-10-06T04:32:24.268930+0000","last_clean":"2023-10-06T04:32:24.268930+0000","last_became_active":"2023-10-06T04:32:24.261681+0000","last_became_peered":"2023-10-06T04:32:24.261681+0000","last_unstale":"2023-10-06T04:32:24.268930+0000","last_undegraded":"2023-10-06T04:32:24.268930+0000","last_fullsized":"2023-10-06T04:32:24.268930+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440007+0000","last_change":"2023-10-06T04:32:23.443656+0000","last_active":"2023-10-06T04:32:24.440007+0000","last_peered":"2023-10-06T04:32:24.440007+0000","last_clean":"2023-10-06T04:32:24.440007+0000","last_became_active":"2023-10-06T04:32:23.443293+0000","last_became_peered":"2023-10-06T04:32:23.443293+0000","last_unstale":"2023-10-06T04:32:24.440007+0000","last_undegraded":"2023-10-06T04:32:24.440007+0000","last_fullsized":"2023-10-06T04:32:24.440007+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270922+0000","last_change":"2023-10-06T04:32:24.264104+0000","last_active":"2023-10-06T04:32:24.270922+0000","last_peered":"2023-10-06T04:32:24.270922+0000","last_clean":"2023-10-06T04:32:24.270922+0000","last_became_active":"2023-10-06T04:32:24.263651+0000","last_became_peered":"2023-10-06T04:32:24.263651+0000","last_unstale":"2023-10-06T04:32:24.270922+0000","last_undegraded":"2023-10-06T04:32:24.270922+0000","last_fullsized":"2023-10-06T04:32:24.270922+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:24.395312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446140+0000","last_change":"2023-10-06T04:32:23.451767+0000","last_active":"2023-10-06T04:32:24.446140+0000","last_peered":"2023-10-06T04:32:24.446140+0000","last_clean":"2023-10-06T04:32:24.446140+0000","last_became_active":"2023-10-06T04:32:23.451430+0000","last_became_peered":"2023-10-06T04:32:23.451430+0000","last_unstale":"2023-10-06T04:32:24.446140+0000","last_undegraded":"2023-10-06T04:32:24.446140+0000","last_fullsized":"2023-10-06T04:32:24.446140+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274735+0000","last_change":"2023-10-06T04:32:24.267750+0000","last_active":"2023-10-06T04:32:24.274735+0000","last_peered":"2023-10-06T04:32:24.274735+0000","last_clean":"2023-10-06T04:32:24.274735+0000","last_became_active":"2023-10-06T04:32:24.265805+0000","last_became_peered":"2023-10-06T04:32:24.265805+0000","last_unstale":"2023-10-06T04:32:24.274735+0000","last_undegraded":"2023-10-06T04:32:24.274735+0000","last_fullsized":"2023-10-06T04:32:24.274735+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251273+0000","last_change":"2023-10-06T04:32:23.449278+0000","last_active":"2023-10-06T04:32:24.251273+0000","last_peered":"2023-10-06T04:32:24.251273+0000","last_clean":"2023-10-06T04:32:24.251273+0000","last_became_active":"2023-10-06T04:32:23.449034+0000","last_became_peered":"2023-10-06T04:32:23.449034+0000","last_unstale":"2023-10-06T04:32:24.251273+0000","last_undegraded":"2023-10-06T04:32:24.251273+0000","last_fullsized":"2023-10-06T04:32:24.251273+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270989+0000","last_change":"2023-10-06T04:32:24.265155+0000","last_active":"2023-10-06T04:32:24.270989+0000","last_peered":"2023-10-06T04:32:24.270989+0000","last_clean":"2023-10-06T04:32:24.270989+0000","last_became_active":"2023-10-06T04:32:24.264863+0000","last_became_peered":"2023-10-06T04:32:24.264863+0000","last_unstale":"2023-10-06T04:32:24.270989+0000","last_undegraded":"2023-10-06T04:32:24.270989+0000","last_fullsized":"2023-10-06T04:32:24.270989+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251056+0000","last_change":"2023-10-06T04:32:23.440295+0000","last_active":"2023-10-06T04:32:24.251056+0000","last_peered":"2023-10-06T04:32:24.251056+0000","last_clean":"2023-10-06T04:32:24.251056+0000","last_became_active":"2023-10-06T04:32:23.440039+0000","last_became_peered":"2023-10-06T04:32:23.440039+0000","last_unstale":"2023-10-06T04:32:24.251056+0000","last_undegraded":"2023-10-06T04:32:24.251056+0000","last_fullsized":"2023-10-06T04:32:24.251056+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.14","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272396+0000","last_change":"2023-10-06T04:32:24.271227+0000","last_active":"2023-10-06T04:32:24.272396+0000","last_peered":"2023-10-06T04:32:24.272396+0000","last_clean":"2023-10-06T04:32:24.272396+0000","last_became_active":"2023-10-06T04:32:24.270984+0000","last_became_peered":"2023-10-06T04:32:24.270984+0000","last_unstale":"2023-10-06T04:32:24.272396+0000","last_undegraded":"2023-10-06T04:32:24.272396+0000","last_fullsized":"2023-10-06T04:32:24.272396+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.260712+0000","last_change":"2023-10-06T04:32:23.447791+0000","last_active":"2023-10-06T04:32:24.260712+0000","last_peered":"2023-10-06T04:32:24.260712+0000","last_clean":"2023-10-06T04:32:24.260712+0000","last_became_active":"2023-10-06T04:32:23.447395+0000","last_became_peered":"2023-10-06T04:32:23.447395+0000","last_unstale":"2023-10-06T04:32:24.260712+0000","last_undegraded":"2023-10-06T04:32:24.260712+0000","last_fullsized":"2023-10-06T04:32:24.260712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:18.839316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274487+0000","last_change":"2023-10-06T04:32:24.269565+0000","last_active":"2023-10-06T04:32:24.274487+0000","last_peered":"2023-10-06T04:32:24.274487+0000","last_clean":"2023-10-06T04:32:24.274487+0000","last_became_active":"2023-10-06T04:32:24.269285+0000","last_became_peered":"2023-10-06T04:32:24.269285+0000","last_unstale":"2023-10-06T04:32:24.274487+0000","last_undegraded":"2023-10-06T04:32:24.274487+0000","last_fullsized":"2023-10-06T04:32:24.274487+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.261745+0000","last_change":"2023-10-06T04:32:23.444329+0000","last_active":"2023-10-06T04:32:24.261745+0000","last_peered":"2023-10-06T04:32:24.261745+0000","last_clean":"2023-10-06T04:32:24.261745+0000","last_became_active":"2023-10-06T04:32:23.444020+0000","last_became_peered":"2023-10-06T04:32:23.444020+0000","last_unstale":"2023-10-06T04:32:24.261745+0000","last_undegraded":"2023-10-06T04:32:24.261745+0000","last_fullsized":"2023-10-06T04:32:24.261745+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.268975+0000","last_change":"2023-10-06T04:32:24.264060+0000","last_active":"2023-10-06T04:32:24.268975+0000","last_peered":"2023-10-06T04:32:24.268975+0000","last_clean":"2023-10-06T04:32:24.268975+0000","last_became_active":"2023-10-06T04:32:24.263825+0000","last_became_peered":"2023-10-06T04:32:24.263825+0000","last_unstale":"2023-10-06T04:32:24.268975+0000","last_undegraded":"2023-10-06T04:32:24.268975+0000","last_fullsized":"2023-10-06T04:32:24.268975+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"33'3","reported_seq":13,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440264+0000","last_change":"2023-10-06T04:32:23.447887+0000","last_active":"2023-10-06T04:32:24.440264+0000","last_peered":"2023-10-06T04:32:24.440264+0000","last_clean":"2023-10-06T04:32:24.440264+0000","last_became_active":"2023-10-06T04:32:23.447650+0000","last_became_peered":"2023-10-06T04:32:23.447650+0000","last_unstale":"2023-10-06T04:32:24.440264+0000","last_undegraded":"2023-10-06T04:32:24.440264+0000","last_fullsized":"2023-10-06T04:32:24.440264+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:47.164678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.277552+0000","last_change":"2023-10-06T04:32:24.270050+0000","last_active":"2023-10-06T04:32:24.277552+0000","last_peered":"2023-10-06T04:32:24.277552+0000","last_clean":"2023-10-06T04:32:24.277552+0000","last_became_active":"2023-10-06T04:32:24.269298+0000","last_became_peered":"2023-10-06T04:32:24.269298+0000","last_unstale":"2023-10-06T04:32:24.277552+0000","last_undegraded":"2023-10-06T04:32:24.277552+0000","last_fullsized":"2023-10-06T04:32:24.277552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"33'6","reported_seq":16,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.447538+0000","last_change":"2023-10-06T04:32:23.447643+0000","last_active":"2023-10-06T04:32:24.447538+0000","last_peered":"2023-10-06T04:32:24.447538+0000","last_clean":"2023-10-06T04:32:24.447538+0000","last_became_active":"2023-10-06T04:32:23.447352+0000","last_became_peered":"2023-10-06T04:32:23.447352+0000","last_unstale":"2023-10-06T04:32:24.447538+0000","last_undegraded":"2023-10-06T04:32:24.447538+0000","last_fullsized":"2023-10-06T04:32:24.447538+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1536,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.10","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274008+0000","last_change":"2023-10-06T04:32:24.267086+0000","last_active":"2023-10-06T04:32:24.274008+0000","last_peered":"2023-10-06T04:32:24.274008+0000","last_clean":"2023-10-06T04:32:24.274008+0000","last_became_active":"2023-10-06T04:32:24.266829+0000","last_became_peered":"2023-10-06T04:32:24.266829+0000","last_unstale":"2023-10-06T04:32:24.274008+0000","last_undegraded":"2023-10-06T04:32:24.274008+0000","last_fullsized":"2023-10-06T04:32:24.274008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440259+0000","last_change":"2023-10-06T04:32:23.443222+0000","last_active":"2023-10-06T04:32:24.440259+0000","last_peered":"2023-10-06T04:32:24.440259+0000","last_clean":"2023-10-06T04:32:24.440259+0000","last_became_active":"2023-10-06T04:32:23.442957+0000","last_became_peered":"2023-10-06T04:32:23.442957+0000","last_unstale":"2023-10-06T04:32:24.440259+0000","last_undegraded":"2023-10-06T04:32:24.440259+0000","last_fullsized":"2023-10-06T04:32:24.440259+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270508+0000","last_change":"2023-10-06T04:32:24.268660+0000","last_active":"2023-10-06T04:32:24.270508+0000","last_peered":"2023-10-06T04:32:24.270508+0000","last_clean":"2023-10-06T04:32:24.270508+0000","last_became_active":"2023-10-06T04:32:24.268397+0000","last_became_peered":"2023-10-06T04:32:24.268397+0000","last_unstale":"2023-10-06T04:32:24.270508+0000","last_undegraded":"2023-10-06T04:32:24.270508+0000","last_fullsized":"2023-10-06T04:32:24.270508+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251966+0000","last_change":"2023-10-06T04:32:23.445203+0000","last_active":"2023-10-06T04:32:24.251966+0000","last_peered":"2023-10-06T04:32:24.251966+0000","last_clean":"2023-10-06T04:32:24.251966+0000","last_became_active":"2023-10-06T04:32:23.444688+0000","last_became_peered":"2023-10-06T04:32:23.444688+0000","last_unstale":"2023-10-06T04:32:24.251966+0000","last_undegraded":"2023-10-06T04:32:24.251966+0000","last_fullsized":"2023-10-06T04:32:24.251966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274578+0000","last_change":"2023-10-06T04:32:24.271729+0000","last_active":"2023-10-06T04:32:24.274578+0000","last_peered":"2023-10-06T04:32:24.274578+0000","last_clean":"2023-10-06T04:32:24.274578+0000","last_became_active":"2023-10-06T04:32:24.271453+0000","last_became_peered":"2023-10-06T04:32:24.271453+0000","last_unstale":"2023-10-06T04:32:24.274578+0000","last_undegraded":"2023-10-06T04:32:24.274578+0000","last_fullsized":"2023-10-06T04:32:24.274578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.259047+0000","last_change":"2023-10-06T04:32:23.440530+0000","last_active":"2023-10-06T04:32:24.259047+0000","last_peered":"2023-10-06T04:32:24.259047+0000","last_clean":"2023-10-06T04:32:24.259047+0000","last_became_active":"2023-10-06T04:32:23.440226+0000","last_became_peered":"2023-10-06T04:32:23.440226+0000","last_unstale":"2023-10-06T04:32:24.259047+0000","last_undegraded":"2023-10-06T04:32:24.259047+0000","last_fullsized":"2023-10-06T04:32:24.259047+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.271024+0000","last_change":"2023-10-06T04:32:24.268363+0000","last_active":"2023-10-06T04:32:24.271024+0000","last_peered":"2023-10-06T04:32:24.271024+0000","last_clean":"2023-10-06T04:32:24.271024+0000","last_became_active":"2023-10-06T04:32:24.268116+0000","last_became_peered":"2023-10-06T04:32:24.268116+0000","last_unstale":"2023-10-06T04:32:24.271024+0000","last_undegraded":"2023-10-06T04:32:24.271024+0000","last_fullsized":"2023-10-06T04:32:24.271024+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"33'8","reported_seq":18,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446724+0000","last_change":"2023-10-06T04:32:23.452500+0000","last_active":"2023-10-06T04:32:24.446724+0000","last_peered":"2023-10-06T04:32:24.446724+0000","last_clean":"2023-10-06T04:32:24.446724+0000","last_became_active":"2023-10-06T04:32:23.452236+0000","last_became_peered":"2023-10-06T04:32:23.452236+0000","last_unstale":"2023-10-06T04:32:24.446724+0000","last_undegraded":"2023-10-06T04:32:24.446724+0000","last_fullsized":"2023-10-06T04:32:24.446724+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270753+0000","last_change":"2023-10-06T04:32:24.265984+0000","last_active":"2023-10-06T04:32:24.270753+0000","last_peered":"2023-10-06T04:32:24.270753+0000","last_clean":"2023-10-06T04:32:24.270753+0000","last_became_active":"2023-10-06T04:32:24.265707+0000","last_became_peered":"2023-10-06T04:32:24.265707+0000","last_unstale":"2023-10-06T04:32:24.270753+0000","last_undegraded":"2023-10-06T04:32:24.270753+0000","last_fullsized":"2023-10-06T04:32:24.270753+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.446333+0000","last_change":"2023-10-06T04:32:23.447020+0000","last_active":"2023-10-06T04:32:24.446333+0000","last_peered":"2023-10-06T04:32:24.446333+0000","last_clean":"2023-10-06T04:32:24.446333+0000","last_became_active":"2023-10-06T04:32:23.446786+0000","last_became_peered":"2023-10-06T04:32:23.446786+0000","last_unstale":"2023-10-06T04:32:24.446333+0000","last_undegraded":"2023-10-06T04:32:24.446333+0000","last_fullsized":"2023-10-06T04:32:24.446333+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.805674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.d","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269631+0000","last_change":"2023-10-06T04:32:24.265722+0000","last_active":"2023-10-06T04:32:24.269631+0000","last_peered":"2023-10-06T04:32:24.269631+0000","last_clean":"2023-10-06T04:32:24.269631+0000","last_became_active":"2023-10-06T04:32:24.265435+0000","last_became_peered":"2023-10-06T04:32:24.265435+0000","last_unstale":"2023-10-06T04:32:24.269631+0000","last_undegraded":"2023-10-06T04:32:24.269631+0000","last_fullsized":"2023-10-06T04:32:24.269631+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.261738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.442797+0000","last_change":"2023-10-06T04:32:23.442348+0000","last_active":"2023-10-06T04:32:24.442797+0000","last_peered":"2023-10-06T04:32:24.442797+0000","last_clean":"2023-10-06T04:32:24.442797+0000","last_became_active":"2023-10-06T04:32:23.441833+0000","last_became_peered":"2023-10-06T04:32:23.441833+0000","last_unstale":"2023-10-06T04:32:24.442797+0000","last_undegraded":"2023-10-06T04:32:24.442797+0000","last_fullsized":"2023-10-06T04:32:24.442797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.916159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.e","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.259088+0000","last_change":"2023-10-06T04:32:24.254622+0000","last_active":"2023-10-06T04:32:24.259088+0000","last_peered":"2023-10-06T04:32:24.259088+0000","last_clean":"2023-10-06T04:32:24.259088+0000","last_became_active":"2023-10-06T04:32:24.254294+0000","last_became_peered":"2023-10-06T04:32:24.254294+0000","last_unstale":"2023-10-06T04:32:24.259088+0000","last_undegraded":"2023-10-06T04:32:24.259088+0000","last_fullsized":"2023-10-06T04:32:24.259088+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"33'1","reported_seq":11,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.443286+0000","last_change":"2023-10-06T04:32:23.447777+0000","last_active":"2023-10-06T04:32:24.443286+0000","last_peered":"2023-10-06T04:32:24.443286+0000","last_clean":"2023-10-06T04:32:24.443286+0000","last_became_active":"2023-10-06T04:32:23.447526+0000","last_became_peered":"2023-10-06T04:32:23.447526+0000","last_unstale":"2023-10-06T04:32:24.443286+0000","last_undegraded":"2023-10-06T04:32:24.443286+0000","last_fullsized":"2023-10-06T04:32:24.443286+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:59.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.272351+0000","last_change":"2023-10-06T04:32:24.270632+0000","last_active":"2023-10-06T04:32:24.272351+0000","last_peered":"2023-10-06T04:32:24.272351+0000","last_clean":"2023-10-06T04:32:24.272351+0000","last_became_active":"2023-10-06T04:32:24.270263+0000","last_became_peered":"2023-10-06T04:32:24.270263+0000","last_unstale":"2023-10-06T04:32:24.272351+0000","last_undegraded":"2023-10-06T04:32:24.272351+0000","last_fullsized":"2023-10-06T04:32:24.272351+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:19.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251541+0000","last_change":"2023-10-06T04:32:23.446504+0000","last_active":"2023-10-06T04:32:24.251541+0000","last_peered":"2023-10-06T04:32:24.251541+0000","last_clean":"2023-10-06T04:32:24.251541+0000","last_became_active":"2023-10-06T04:32:23.446221+0000","last_became_peered":"2023-10-06T04:32:23.446221+0000","last_unstale":"2023-10-06T04:32:24.251541+0000","last_undegraded":"2023-10-06T04:32:24.251541+0000","last_fullsized":"2023-10-06T04:32:24.251541+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274551+0000","last_change":"2023-10-06T04:32:24.271430+0000","last_active":"2023-10-06T04:32:24.274551+0000","last_peered":"2023-10-06T04:32:24.274551+0000","last_clean":"2023-10-06T04:32:24.274551+0000","last_became_active":"2023-10-06T04:32:24.271180+0000","last_became_peered":"2023-10-06T04:32:24.271180+0000","last_unstale":"2023-10-06T04:32:24.274551+0000","last_undegraded":"2023-10-06T04:32:24.274551+0000","last_fullsized":"2023-10-06T04:32:24.274551+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.439980+0000","last_change":"2023-10-06T04:32:23.445911+0000","last_active":"2023-10-06T04:32:24.439980+0000","last_peered":"2023-10-06T04:32:24.439980+0000","last_clean":"2023-10-06T04:32:24.439980+0000","last_became_active":"2023-10-06T04:32:23.445582+0000","last_became_peered":"2023-10-06T04:32:23.445582+0000","last_unstale":"2023-10-06T04:32:24.439980+0000","last_undegraded":"2023-10-06T04:32:24.439980+0000","last_fullsized":"2023-10-06T04:32:24.439980+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:35.404671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.267532+0000","last_change":"2023-10-06T04:32:24.259250+0000","last_active":"2023-10-06T04:32:24.267532+0000","last_peered":"2023-10-06T04:32:24.267532+0000","last_clean":"2023-10-06T04:32:24.267532+0000","last_became_active":"2023-10-06T04:32:24.258962+0000","last_became_peered":"2023-10-06T04:32:24.258962+0000","last_unstale":"2023-10-06T04:32:24.267532+0000","last_undegraded":"2023-10-06T04:32:24.267532+0000","last_fullsized":"2023-10-06T04:32:24.267532+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257714+0000","last_change":"2023-10-06T04:32:23.445529+0000","last_active":"2023-10-06T04:32:24.257714+0000","last_peered":"2023-10-06T04:32:24.257714+0000","last_clean":"2023-10-06T04:32:24.257714+0000","last_became_active":"2023-10-06T04:32:23.445245+0000","last_became_peered":"2023-10-06T04:32:23.445245+0000","last_unstale":"2023-10-06T04:32:24.257714+0000","last_undegraded":"2023-10-06T04:32:24.257714+0000","last_fullsized":"2023-10-06T04:32:24.257714+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.277300+0000","last_change":"2023-10-06T04:32:24.270658+0000","last_active":"2023-10-06T04:32:24.277300+0000","last_peered":"2023-10-06T04:32:24.277300+0000","last_clean":"2023-10-06T04:32:24.277300+0000","last_became_active":"2023-10-06T04:32:24.270394+0000","last_became_peered":"2023-10-06T04:32:24.270394+0000","last_unstale":"2023-10-06T04:32:24.277300+0000","last_undegraded":"2023-10-06T04:32:24.277300+0000","last_fullsized":"2023-10-06T04:32:24.277300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.256385+0000","last_change":"2023-10-06T04:32:23.452178+0000","last_active":"2023-10-06T04:32:24.256385+0000","last_peered":"2023-10-06T04:32:24.256385+0000","last_clean":"2023-10-06T04:32:24.256385+0000","last_became_active":"2023-10-06T04:32:23.451849+0000","last_became_peered":"2023-10-06T04:32:23.451849+0000","last_unstale":"2023-10-06T04:32:24.256385+0000","last_undegraded":"2023-10-06T04:32:24.256385+0000","last_fullsized":"2023-10-06T04:32:24.256385+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.026875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.b","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.271008+0000","last_change":"2023-10-06T04:32:24.265320+0000","last_active":"2023-10-06T04:32:24.271008+0000","last_peered":"2023-10-06T04:32:24.271008+0000","last_clean":"2023-10-06T04:32:24.271008+0000","last_became_active":"2023-10-06T04:32:24.264894+0000","last_became_peered":"2023-10-06T04:32:24.264894+0000","last_unstale":"2023-10-06T04:32:24.271008+0000","last_undegraded":"2023-10-06T04:32:24.271008+0000","last_fullsized":"2023-10-06T04:32:24.271008+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.258192+0000","last_change":"2023-10-06T04:32:23.444136+0000","last_active":"2023-10-06T04:32:24.258192+0000","last_peered":"2023-10-06T04:32:24.258192+0000","last_clean":"2023-10-06T04:32:24.258192+0000","last_became_active":"2023-10-06T04:32:23.443839+0000","last_became_peered":"2023-10-06T04:32:23.443839+0000","last_unstale":"2023-10-06T04:32:24.258192+0000","last_undegraded":"2023-10-06T04:32:24.258192+0000","last_fullsized":"2023-10-06T04:32:24.258192+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:10.814479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.276784+0000","last_change":"2023-10-06T04:32:24.269672+0000","last_active":"2023-10-06T04:32:24.276784+0000","last_peered":"2023-10-06T04:32:24.276784+0000","last_clean":"2023-10-06T04:32:24.276784+0000","last_became_active":"2023-10-06T04:32:24.269407+0000","last_became_peered":"2023-10-06T04:32:24.269407+0000","last_unstale":"2023-10-06T04:32:24.276784+0000","last_undegraded":"2023-10-06T04:32:24.276784+0000","last_fullsized":"2023-10-06T04:32:24.276784+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.252318+0000","last_change":"2023-10-06T04:32:23.446685+0000","last_active":"2023-10-06T04:32:24.252318+0000","last_peered":"2023-10-06T04:32:24.252318+0000","last_clean":"2023-10-06T04:32:24.252318+0000","last_became_active":"2023-10-06T04:32:23.446432+0000","last_became_peered":"2023-10-06T04:32:23.446432+0000","last_unstale":"2023-10-06T04:32:24.252318+0000","last_undegraded":"2023-10-06T04:32:24.252318+0000","last_fullsized":"2023-10-06T04:32:24.252318+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251619+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:32:24.251619+0000","last_peered":"2023-10-06T04:32:24.251619+0000","last_clean":"2023-10-06T04:32:24.251619+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:32:24.251619+0000","last_undegraded":"2023-10-06T04:32:24.251619+0000","last_fullsized":"2023-10-06T04:32:24.251619+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270201+0000","last_change":"2023-10-06T04:32:24.266084+0000","last_active":"2023-10-06T04:32:24.270201+0000","last_peered":"2023-10-06T04:32:24.270201+0000","last_clean":"2023-10-06T04:32:24.270201+0000","last_became_active":"2023-10-06T04:32:24.265847+0000","last_became_peered":"2023-10-06T04:32:24.265847+0000","last_unstale":"2023-10-06T04:32:24.270201+0000","last_undegraded":"2023-10-06T04:32:24.270201+0000","last_fullsized":"2023-10-06T04:32:24.270201+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.252731+0000","last_change":"2023-10-06T04:32:23.451106+0000","last_active":"2023-10-06T04:32:24.252731+0000","last_peered":"2023-10-06T04:32:24.252731+0000","last_clean":"2023-10-06T04:32:24.252731+0000","last_became_active":"2023-10-06T04:32:23.450838+0000","last_became_peered":"2023-10-06T04:32:23.450838+0000","last_unstale":"2023-10-06T04:32:24.252731+0000","last_undegraded":"2023-10-06T04:32:24.252731+0000","last_fullsized":"2023-10-06T04:32:24.252731+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.261318+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:32:24.261318+0000","last_peered":"2023-10-06T04:32:24.261318+0000","last_clean":"2023-10-06T04:32:24.261318+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:32:24.261318+0000","last_undegraded":"2023-10-06T04:32:24.261318+0000","last_fullsized":"2023-10-06T04:32:24.261318+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274657+0000","last_change":"2023-10-06T04:32:24.272289+0000","last_active":"2023-10-06T04:32:24.274657+0000","last_peered":"2023-10-06T04:32:24.274657+0000","last_clean":"2023-10-06T04:32:24.274657+0000","last_became_active":"2023-10-06T04:32:24.271996+0000","last_became_peered":"2023-10-06T04:32:24.271996+0000","last_unstale":"2023-10-06T04:32:24.274657+0000","last_undegraded":"2023-10-06T04:32:24.274657+0000","last_fullsized":"2023-10-06T04:32:24.274657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.256443+0000","last_change":"2023-10-06T04:32:23.444286+0000","last_active":"2023-10-06T04:32:24.256443+0000","last_peered":"2023-10-06T04:32:24.256443+0000","last_clean":"2023-10-06T04:32:24.256443+0000","last_became_active":"2023-10-06T04:32:23.443982+0000","last_became_peered":"2023-10-06T04:32:23.443982+0000","last_unstale":"2023-10-06T04:32:24.256443+0000","last_undegraded":"2023-10-06T04:32:24.256443+0000","last_fullsized":"2023-10-06T04:32:24.256443+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:20.394340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257440+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:32:24.257440+0000","last_peered":"2023-10-06T04:32:24.257440+0000","last_clean":"2023-10-06T04:32:24.257440+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:32:24.257440+0000","last_undegraded":"2023-10-06T04:32:24.257440+0000","last_fullsized":"2023-10-06T04:32:24.257440+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.7","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.269441+0000","last_change":"2023-10-06T04:32:24.265180+0000","last_active":"2023-10-06T04:32:24.269441+0000","last_peered":"2023-10-06T04:32:24.269441+0000","last_clean":"2023-10-06T04:32:24.269441+0000","last_became_active":"2023-10-06T04:32:24.264769+0000","last_became_peered":"2023-10-06T04:32:24.264769+0000","last_unstale":"2023-10-06T04:32:24.269441+0000","last_undegraded":"2023-10-06T04:32:24.269441+0000","last_fullsized":"2023-10-06T04:32:24.269441+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257145+0000","last_change":"2023-10-06T04:32:23.452281+0000","last_active":"2023-10-06T04:32:24.257145+0000","last_peered":"2023-10-06T04:32:24.257145+0000","last_clean":"2023-10-06T04:32:24.257145+0000","last_became_active":"2023-10-06T04:32:23.452023+0000","last_became_peered":"2023-10-06T04:32:23.452023+0000","last_unstale":"2023-10-06T04:32:24.257145+0000","last_undegraded":"2023-10-06T04:32:24.257145+0000","last_fullsized":"2023-10-06T04:32:24.257145+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":21,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.261356+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:32:24.261356+0000","last_peered":"2023-10-06T04:32:24.261356+0000","last_clean":"2023-10-06T04:32:24.261356+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:32:24.261356+0000","last_undegraded":"2023-10-06T04:32:24.261356+0000","last_fullsized":"2023-10-06T04:32:24.261356+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":175,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257561+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:32:24.257561+0000","last_peered":"2023-10-06T04:32:24.257561+0000","last_clean":"2023-10-06T04:32:24.257561+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:32:24.257561+0000","last_undegraded":"2023-10-06T04:32:24.257561+0000","last_fullsized":"2023-10-06T04:32:24.257561+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270305+0000","last_change":"2023-10-06T04:32:24.267047+0000","last_active":"2023-10-06T04:32:24.270305+0000","last_peered":"2023-10-06T04:32:24.270305+0000","last_clean":"2023-10-06T04:32:24.270305+0000","last_became_active":"2023-10-06T04:32:24.266777+0000","last_became_peered":"2023-10-06T04:32:24.266777+0000","last_unstale":"2023-10-06T04:32:24.270305+0000","last_undegraded":"2023-10-06T04:32:24.270305+0000","last_fullsized":"2023-10-06T04:32:24.270305+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"33'2","reported_seq":12,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.440198+0000","last_change":"2023-10-06T04:32:23.448480+0000","last_active":"2023-10-06T04:32:24.440198+0000","last_peered":"2023-10-06T04:32:24.440198+0000","last_clean":"2023-10-06T04:32:24.440198+0000","last_became_active":"2023-10-06T04:32:23.448242+0000","last_became_peered":"2023-10-06T04:32:23.448242+0000","last_unstale":"2023-10-06T04:32:24.440198+0000","last_undegraded":"2023-10-06T04:32:24.440198+0000","last_fullsized":"2023-10-06T04:32:24.440198+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257653+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:32:24.257653+0000","last_peered":"2023-10-06T04:32:24.257653+0000","last_clean":"2023-10-06T04:32:24.257653+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:32:24.257653+0000","last_undegraded":"2023-10-06T04:32:24.257653+0000","last_fullsized":"2023-10-06T04:32:24.257653+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.270236+0000","last_change":"2023-10-06T04:32:24.266050+0000","last_active":"2023-10-06T04:32:24.270236+0000","last_peered":"2023-10-06T04:32:24.270236+0000","last_clean":"2023-10-06T04:32:24.270236+0000","last_became_active":"2023-10-06T04:32:24.265769+0000","last_became_peered":"2023-10-06T04:32:24.265769+0000","last_unstale":"2023-10-06T04:32:24.270236+0000","last_undegraded":"2023-10-06T04:32:24.270236+0000","last_fullsized":"2023-10-06T04:32:24.270236+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:01.427293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.263759+0000","last_change":"2023-10-06T04:32:23.443114+0000","last_active":"2023-10-06T04:32:24.263759+0000","last_peered":"2023-10-06T04:32:24.263759+0000","last_clean":"2023-10-06T04:32:24.263759+0000","last_became_active":"2023-10-06T04:32:23.442844+0000","last_became_peered":"2023-10-06T04:32:23.442844+0000","last_unstale":"2023-10-06T04:32:24.263759+0000","last_undegraded":"2023-10-06T04:32:24.263759+0000","last_fullsized":"2023-10-06T04:32:24.263759+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.257458+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:32:24.257458+0000","last_peered":"2023-10-06T04:32:24.257458+0000","last_clean":"2023-10-06T04:32:24.257458+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:32:24.257458+0000","last_undegraded":"2023-10-06T04:32:24.257458+0000","last_fullsized":"2023-10-06T04:32:24.257458+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274623+0000","last_change":"2023-10-06T04:32:24.265505+0000","last_active":"2023-10-06T04:32:24.274623+0000","last_peered":"2023-10-06T04:32:24.274623+0000","last_clean":"2023-10-06T04:32:24.274623+0000","last_became_active":"2023-10-06T04:32:24.265228+0000","last_became_peered":"2023-10-06T04:32:24.265228+0000","last_unstale":"2023-10-06T04:32:24.274623+0000","last_undegraded":"2023-10-06T04:32:24.274623+0000","last_fullsized":"2023-10-06T04:32:24.274623+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.250899+0000","last_change":"2023-10-06T04:32:23.438229+0000","last_active":"2023-10-06T04:32:24.250899+0000","last_peered":"2023-10-06T04:32:24.250899+0000","last_clean":"2023-10-06T04:32:24.250899+0000","last_became_active":"2023-10-06T04:32:23.437954+0000","last_became_peered":"2023-10-06T04:32:23.437954+0000","last_unstale":"2023-10-06T04:32:24.250899+0000","last_undegraded":"2023-10-06T04:32:24.250899+0000","last_fullsized":"2023-10-06T04:32:24.250899+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:22.972008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":23,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.251970+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:32:24.251970+0000","last_peered":"2023-10-06T04:32:24.251970+0000","last_clean":"2023-10-06T04:32:24.251970+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:32:24.251970+0000","last_undegraded":"2023-10-06T04:32:24.251970+0000","last_fullsized":"2023-10-06T04:32:24.251970+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":9,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.274648+0000","last_change":"2023-10-06T04:32:24.272223+0000","last_active":"2023-10-06T04:32:24.274648+0000","last_peered":"2023-10-06T04:32:24.274648+0000","last_clean":"2023-10-06T04:32:24.274648+0000","last_became_active":"2023-10-06T04:32:24.271905+0000","last_became_peered":"2023-10-06T04:32:24.271905+0000","last_unstale":"2023-10-06T04:32:24.274648+0000","last_undegraded":"2023-10-06T04:32:24.274648+0000","last_fullsized":"2023-10-06T04:32:24.274648+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:17.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.260527+0000","last_change":"2023-10-06T04:32:23.444321+0000","last_active":"2023-10-06T04:32:24.260527+0000","last_peered":"2023-10-06T04:32:24.260527+0000","last_clean":"2023-10-06T04:32:24.260527+0000","last_became_active":"2023-10-06T04:32:23.444073+0000","last_became_peered":"2023-10-06T04:32:23.444073+0000","last_unstale":"2023-10-06T04:32:24.260527+0000","last_undegraded":"2023-10-06T04:32:24.260527+0000","last_fullsized":"2023-10-06T04:32:24.260527+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:37.675115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":22,"reported_epoch":33,"state":"active+clean","last_fresh":"2023-10-06T04:32:24.262534+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:32:24.262534+0000","last_peered":"2023-10-06T04:32:24.262534+0000","last_clean":"2023-10-06T04:32:24.262534+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:32:24.262534+0000","last_undegraded":"2023-10-06T04:32:24.262534+0000","last_fullsized":"2023-10-06T04:32:24.262534+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":2306,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":13,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574865,"num_pgs":8,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298248,"kb_used_data":960,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073592,"statfs":{"total":96636764160,"available":96331358208,"internally_reserved":0,"allocated":983040,"data_stored":741769,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.63300000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.67000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.65300000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.624},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.69499999999999995}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.501}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.57199999999999995}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.68500000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":150349,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.746},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.67500000000000004}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.86399999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.78400000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.495}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.40699999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.60099999999999998}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149885,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.36899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.32500000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.39900000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.434},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.35999999999999999}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.753}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.67000000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.71999999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.73099999999999998},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.64100000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298236,"kb_used_data":948,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073604,"statfs":{"total":96636764160,"available":96331370496,"internally_reserved":0,"allocated":970752,"data_stored":740197,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.24399999999999999},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.26700000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.33800000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.28999999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.32300000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.42099999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.434},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.41099999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.49199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.51700000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56999999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574865,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297656,"kb_used_data":368,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074184,"statfs":{"total":96636764160,"available":96331964416,"internally_reserved":0,"allocated":376832,"data_stored":149865,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.30199999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.44700000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.39800000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.436}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.50800000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.60799999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.68999999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297680,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074160,"statfs":{"total":96636764160,"available":96331939840,"internally_reserved":0,"allocated":401408,"data_stored":158006,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.24399999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.22900000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.32100000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.53900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.497}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.57699999999999996}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":150371,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.437},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.36699999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.48799999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.41499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.55300000000000005}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.58299999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.627},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.63900000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297660,"kb_used_data":372,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074180,"statfs":{"total":96636764160,"available":96331960320,"internally_reserved":0,"allocated":380928,"data_stored":149907,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.44700000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.46400000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.375},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.54200000000000004}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.503},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.56000000000000005}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.625}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.57899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.60699999999999998}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":554,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1606,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:32:28.880 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:32:28.880 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-10-06T04:32:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.031+0000 7fde6fd5c700 1 Processor -- start 2023-10-06T04:32:29.034 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.032+0000 7fde6fd5c700 1 -- start start 2023-10-06T04:32:29.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.032+0000 7fde6fd5c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde680688c0 0x7fde680ab710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.036 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.032+0000 7fde6fd5c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde680abc50 0x7fde680c8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.032+0000 7fde6fd5c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde680ac0d0 con 0x7fde68068c90 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.032+0000 7fde6fd5c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde680c6de0 con 0x7fde680abc50 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.032+0000 7fde6fd5c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde680c70c0 con 0x7fde680688c0 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6e2f9700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde680abc50 0x7fde680c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6d2f7700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fde68068c90 0x7fde680c7e70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40422/0 (socket says 172.21.15.123:40422) 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6d2f7700 1 -- 172.21.15.123:0/2189995227 learned_addr learned my addr 172.21.15.123:0/2189995227 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6daf8700 1 --2- 172.21.15.123:0/2189995227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde680688c0 0x7fde680ab710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6e2f9700 1 -- 172.21.15.123:0/2189995227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde680688c0 msgr2=0x7fde680ab710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6e2f9700 1 --2- 172.21.15.123:0/2189995227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde680688c0 0x7fde680ab710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6e2f9700 1 -- 172.21.15.123:0/2189995227 >> v1:172.21.15.123:6789/0 conn(0x7fde68068c90 legacy=0x7fde680c7e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6e2f9700 1 -- 172.21.15.123:0/2189995227 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde680c73a0 con 0x7fde680abc50 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6caf6700 1 -- 172.21.15.123:0/2189995227 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3441923162 0 0) 0x7fde680ac0d0 con 0x7fde68068c90 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6e2f9700 1 --2- 172.21.15.123:0/2189995227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680abc50 0x7fde680c8580 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fde5800af70 tx=0x7fde5800a890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8290d0bbd8eba28 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6caf6700 1 -- 172.21.15.123:0/2189995227 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5800f070 con 0x7fde680abc50 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6caf6700 1 -- 172.21.15.123:0/2189995227 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5800a2e0 con 0x7fde680abc50 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.033+0000 7fde6caf6700 1 -- 172.21.15.123:0/2189995227 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5800cd20 con 0x7fde680abc50 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 -- 172.21.15.123:0/2189995227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680abc50 msgr2=0x7fde680c8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/2189995227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680abc50 0x7fde680c8580 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7fde5800af70 tx=0x7fde5800a890 comp rx=0 tx=0).stop 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 -- 172.21.15.123:0/2189995227 shutdown_connections 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/2189995227 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680abc50 0x7fde680c8580 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/2189995227 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde680688c0 0x7fde680ab710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 -- 172.21.15.123:0/2189995227 >> 172.21.15.123:0/2189995227 conn(0x7fde680c9ca0 msgr2=0x7fde6806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:29.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 -- 172.21.15.123:0/2189995227 shutdown_connections 2023-10-06T04:32:29.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.034+0000 7fde6fd5c700 1 -- 172.21.15.123:0/2189995227 wait complete. 2023-10-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.035+0000 7fde6fd5c700 1 Processor -- start 2023-10-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.035+0000 7fde6fd5c700 1 -- start start 2023-10-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6fd5c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680688c0 0x7fde68151760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6fd5c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde68068c90 0x7fde6814ebe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6fd5c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde680abc50 0x7fde68153b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6fd5c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde680ac4e0 con 0x7fde680688c0 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6fd5c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde68151240 con 0x7fde680abc50 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6fd5c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde6814f120 con 0x7fde68068c90 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680688c0 0x7fde68151760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680688c0 0x7fde68151760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35900/0 (socket says 172.21.15.123:35900) 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 -- 172.21.15.123:0/3894853902 learned_addr learned my addr 172.21.15.123:0/3894853902 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6d2f7700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde68068c90 0x7fde6814ebe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6e2f9700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde680abc50 0x7fde68153b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 -- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde68068c90 msgr2=0x7fde6814ebe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde68068c90 0x7fde6814ebe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 -- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde680abc50 msgr2=0x7fde68153b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde680abc50 0x7fde68153b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 -- 172.21.15.123:0/3894853902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde5800ab30 con 0x7fde680688c0 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6d2f7700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde68068c90 0x7fde6814ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6e2f9700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde680abc50 0x7fde68153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde6daf8700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680688c0 0x7fde68151760 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d8c0 tx=0x7fde60008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde65ffb700 1 -- 172.21.15.123:0/3894853902 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000de60 con 0x7fde680688c0 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.036+0000 7fde65ffb700 1 -- 172.21.15.123:0/3894853902 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde6000b780 con 0x7fde680688c0 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.037+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde68169260 con 0x7fde680688c0 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.037+0000 7fde65ffb700 1 -- 172.21.15.123:0/3894853902 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde6000bb20 con 0x7fde680688c0 2023-10-06T04:32:29.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.037+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde68169820 con 0x7fde680688c0 2023-10-06T04:32:29.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.038+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6804bb80 con 0x7fde680688c0 2023-10-06T04:32:29.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.038+0000 7fde65ffb700 1 -- 172.21.15.123:0/3894853902 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde6001e460 con 0x7fde680688c0 2023-10-06T04:32:29.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.038+0000 7fde65ffb700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde54066f30 0x7fde540693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.038+0000 7fde65ffb700 1 -- 172.21.15.123:0/3894853902 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fde60084cd0 con 0x7fde680688c0 2023-10-06T04:32:29.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.038+0000 7fde6d2f7700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde54066f30 0x7fde540693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.039+0000 7fde6d2f7700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde54066f30 0x7fde540693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde68168cc0 tx=0x7fde5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:29.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.043+0000 7fde65ffb700 1 -- 172.21.15.123:0/3894853902 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde60051de0 con 0x7fde680688c0 2023-10-06T04:32:29.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.298+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fde6806e990 con 0x7fde680688c0 2023-10-06T04:32:29.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.301+0000 7fde65ffb700 1 -- 172.21.15.123:0/3894853902 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v9) v1 ==== 190+0+0 (secure 0 0 0) 0x7fde60055400 con 0x7fde680688c0 2023-10-06T04:32:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde54066f30 msgr2=0x7fde540693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde54066f30 0x7fde540693e0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fde68168cc0 tx=0x7fde5c009040 comp rx=0 tx=0).stop 2023-10-06T04:32:29.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680688c0 msgr2=0x7fde68151760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680688c0 0x7fde68151760 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7fde6000d8c0 tx=0x7fde60008040 comp rx=0 tx=0).stop 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 shutdown_connections 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde54066f30 0x7fde540693e0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde680688c0 0x7fde68151760 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde680abc50 0x7fde68153b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 --2- 172.21.15.123:0/3894853902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde68068c90 0x7fde6814ebe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 >> 172.21.15.123:0/3894853902 conn(0x7fde680c9ca0 msgr2=0x7fde68158e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 shutdown_connections 2023-10-06T04:32:29.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.304+0000 7fde6fd5c700 1 -- 172.21.15.123:0/3894853902 wait complete. 2023-10-06T04:32:29.308 INFO:teuthology.orchestra.run.smithi123.stderr:updated caps for client.0 2023-10-06T04:32:29.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:29.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.432+0000 7f873b5fc700 1 Processor -- start 2023-10-06T04:32:29.435 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.432+0000 7f873b5fc700 1 -- start start 2023-10-06T04:32:29.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f873b5fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8734067660 0x7f8734067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f873b5fc700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f87340c9570 0x7f873406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f873b5fc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8734067f70 con 0x7f87340ca5b0 2023-10-06T04:32:29.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f873b5fc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873406a750 con 0x7f8734067660 2023-10-06T04:32:29.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f873b5fc700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f873406aa30 con 0x7f87340c9570 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f8739398700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8734067660 0x7f8734067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f8739b99700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f87340ca5b0 0x7f873406e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40424/0 (socket says 172.21.15.123:40424) 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f8739b99700 1 -- 172.21.15.123:0/3270026621 learned_addr learned my addr 172.21.15.123:0/3270026621 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.433+0000 7f8738b97700 1 --2- 172.21.15.123:0/3270026621 >> v2:172.21.15.177:3300/0 conn(0x7f87340c9570 0x7f873406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f8739398700 1 -- 172.21.15.123:0/3270026621 >> v2:172.21.15.177:3300/0 conn(0x7f87340c9570 msgr2=0x7f873406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f8739398700 1 --2- 172.21.15.123:0/3270026621 >> v2:172.21.15.177:3300/0 conn(0x7f87340c9570 0x7f873406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f8739398700 1 -- 172.21.15.123:0/3270026621 >> v1:172.21.15.123:6789/0 conn(0x7f87340ca5b0 legacy=0x7f873406e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f872bfff700 1 -- 172.21.15.123:0/3270026621 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3952592445 0 0) 0x7f8734067f70 con 0x7f87340ca5b0 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f8739398700 1 -- 172.21.15.123:0/3270026621 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f873406ad10 con 0x7f8734067660 2023-10-06T04:32:29.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f8738b97700 1 --2- 172.21.15.123:0/3270026621 >> v2:172.21.15.177:3300/0 conn(0x7f87340c9570 0x7f873406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f8739398700 1 --2- 172.21.15.123:0/3270026621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8734067660 0x7f8734067a30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f873406fe40 tx=0x7f87240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6ceb8c8005380b3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f872bfff700 1 -- 172.21.15.123:0/3270026621 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400b6b0 con 0x7f8734067660 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f872bfff700 1 -- 172.21.15.123:0/3270026621 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872400b810 con 0x7f8734067660 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f872bfff700 1 -- 172.21.15.123:0/3270026621 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872400bbb0 con 0x7f8734067660 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f873b5fc700 1 -- 172.21.15.123:0/3270026621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8734067660 msgr2=0x7f8734067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f873b5fc700 1 --2- 172.21.15.123:0/3270026621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8734067660 0x7f8734067a30 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f873406fe40 tx=0x7f87240091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f873b5fc700 1 -- 172.21.15.123:0/3270026621 shutdown_connections 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f873b5fc700 1 --2- 172.21.15.123:0/3270026621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8734067660 0x7f8734067a30 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f873b5fc700 1 --2- 172.21.15.123:0/3270026621 >> v2:172.21.15.177:3300/0 conn(0x7f87340c9570 0x7f873406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.434+0000 7f873b5fc700 1 -- 172.21.15.123:0/3270026621 >> 172.21.15.123:0/3270026621 conn(0x7f87340cda80 msgr2=0x7f87340c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.435+0000 7f873b5fc700 1 -- 172.21.15.123:0/3270026621 shutdown_connections 2023-10-06T04:32:29.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.435+0000 7f873b5fc700 1 -- 172.21.15.123:0/3270026621 wait complete. 2023-10-06T04:32:29.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 Processor -- start 2023-10-06T04:32:29.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 -- start start 2023-10-06T04:32:29.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8734067660 0x7f87340af0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87340c9570 0x7f873416d170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87340ca5b0 0x7f87341561a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f873416ddb0 con 0x7f87340ca5b0 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87340ae210 con 0x7f87340c9570 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.436+0000 7f873b5fc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87340ad600 con 0x7f8734067660 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87340ca5b0 0x7f87341561a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87340ca5b0 0x7f87341561a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35912/0 (socket says 172.21.15.123:35912) 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 -- 172.21.15.123:0/28795338 learned_addr learned my addr 172.21.15.123:0/28795338 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8738b97700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87340c9570 0x7f873416d170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739398700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8734067660 0x7f87340af0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 -- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8734067660 msgr2=0x7f87340af0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8734067660 0x7f87340af0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 -- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87340c9570 msgr2=0x7f873416d170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87340c9570 0x7f873416d170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 -- 172.21.15.123:0/28795338 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8724009420 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f8739b99700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87340ca5b0 0x7f87341561a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f872c00cbf0 tx=0x7f872c0019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f87297fa700 1 -- 172.21.15.123:0/28795338 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00a8b0 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f87297fa700 1 -- 172.21.15.123:0/28795338 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872c00aa10 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f87297fa700 1 -- 172.21.15.123:0/28795338 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872c00add0 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87340ad8e0 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.437+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87340adea0 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.438+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87340ae3c0 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.439+0000 7f87297fa700 1 -- 172.21.15.123:0/28795338 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f872c007300 con 0x7f87340ca5b0 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.439+0000 7f87297fa700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8718066f30 0x7f87180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:29.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.439+0000 7f87297fa700 1 -- 172.21.15.123:0/28795338 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f872c0848d0 con 0x7f87340ca5b0 2023-10-06T04:32:29.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.439+0000 7f8739398700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8718066f30 0x7f87180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:29.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.439+0000 7f8739398700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8718066f30 0x7f87180693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87240093f0 tx=0x7f8724017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:29.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.443+0000 7f87297fa700 1 -- 172.21.15.123:0/28795338 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f87340ae3c0 con 0x7f87340ca5b0 2023-10-06T04:32:29.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.667+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87340ae380 con 0x7f87340ca5b0 2023-10-06T04:32:29.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.668+0000 7f87297fa700 1 -- 172.21.15.123:0/28795338 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f872c051a60 con 0x7f87340ca5b0 2023-10-06T04:32:29.670 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:29.670 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2836433084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2836433084},{"type":"v1","addr":"172.21.15.177:6834","nonce":2836433084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3003404474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3003404474},{"type":"v1","addr":"172.21.15.123:6836","nonce":3003404474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1646131544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1646131544},{"type":"v1","addr":"172.21.15.123:6837","nonce":1646131544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:27.453433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/1099021797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1099021797},{"type":"v1","addr":"172.21.15.177:6835","nonce":1099021797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:32:29.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8718066f30 msgr2=0x7f87180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8718066f30 0x7f87180693e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f87240093f0 tx=0x7f8724017000 comp rx=0 tx=0).stop 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87340ca5b0 msgr2=0x7f87341561a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87340ca5b0 0x7f87341561a0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f872c00cbf0 tx=0x7f872c0019b0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 shutdown_connections 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8718066f30 0x7f87180693e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87340ca5b0 0x7f87341561a0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8734067660 0x7f87340af0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 --2- 172.21.15.123:0/28795338 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87340c9570 0x7f873416d170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 >> 172.21.15.123:0/28795338 conn(0x7f87340cda80 msgr2=0x7f8734158480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 shutdown_connections 2023-10-06T04:32:29.673 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:29.670+0000 7f873b5fc700 1 -- 172.21.15.123:0/28795338 wait complete. 2023-10-06T04:32:29.674 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 17 2023-10-06T04:32:30.086 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:32:23.822729+0000', 'modified': '2023-10-06T04:32:27.453433+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'd', 'rank': 0, 'incarnation': 13, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.177:6835/1099021797', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.177:6833', 'nonce': 1099021797}, {'type': 'v1', 'addr': '172.21.15.177:6835', 'nonce': 1099021797}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 5, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:32:30.087 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:32:30.087 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-10-06T04:32:30.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.206+0000 7f1a57937700 1 Processor -- start 2023-10-06T04:32:30.208 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.206+0000 7f1a57937700 1 -- start start 2023-10-06T04:32:30.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a57937700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a50142900 0x7f1a500c75b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a57937700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a50068270 con 0x7f1a5006e960 2023-10-06T04:32:30.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a57937700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1a501597f0 con 0x7f1a5015db30 2023-10-06T04:32:30.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a57937700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1a50067660 con 0x7f1a50142900 2023-10-06T04:32:30.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a556d3700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1a50142900 0x7f1a500c75b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a55ed4700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1a5006e960 0x7f1a50068700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:40426/0 (socket says 172.21.15.123:40426) 2023-10-06T04:32:30.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a55ed4700 1 -- 172.21.15.123:0/1491799709 learned_addr learned my addr 172.21.15.123:0/1491799709 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a47fff700 1 -- 172.21.15.123:0/1491799709 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4002033593 0 0) 0x7f1a50068270 con 0x7f1a5006e960 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a47fff700 1 -- 172.21.15.123:0/1491799709 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1a3c003570 con 0x7f1a5006e960 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a556d3700 1 -- 172.21.15.123:0/1491799709 >> v1:172.21.15.177:6789/0 conn(0x7f1a5015db30 legacy=0x7f1a5006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a556d3700 1 -- 172.21.15.123:0/1491799709 >> v1:172.21.15.123:6789/0 conn(0x7f1a5006e960 legacy=0x7f1a50068700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a556d3700 1 -- 172.21.15.123:0/1491799709 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a50067940 con 0x7f1a50142900 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.207+0000 7f1a556d3700 1 --2- 172.21.15.123:0/1491799709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c75b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098e0 tx=0x7f1a4c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25982fa0be5d8fc1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a47fff700 1 -- 172.21.15.123:0/1491799709 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00b640 con 0x7f1a50142900 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a47fff700 1 -- 172.21.15.123:0/1491799709 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c00b7a0 con 0x7f1a50142900 2023-10-06T04:32:30.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a47fff700 1 -- 172.21.15.123:0/1491799709 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c00bb40 con 0x7f1a50142900 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a57937700 1 -- 172.21.15.123:0/1491799709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 msgr2=0x7f1a500c75b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a57937700 1 --2- 172.21.15.123:0/1491799709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c75b0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098e0 tx=0x7f1a4c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a57937700 1 -- 172.21.15.123:0/1491799709 shutdown_connections 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a57937700 1 --2- 172.21.15.123:0/1491799709 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c75b0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a57937700 1 -- 172.21.15.123:0/1491799709 >> 172.21.15.123:0/1491799709 conn(0x7f1a500cd010 msgr2=0x7f1a500cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a57937700 1 -- 172.21.15.123:0/1491799709 shutdown_connections 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.208+0000 7f1a57937700 1 -- 172.21.15.123:0/1491799709 wait complete. 2023-10-06T04:32:30.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.209+0000 7f1a57937700 1 Processor -- start 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.209+0000 7f1a57937700 1 -- start start 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a57937700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a5006e960 0x7f1a50157ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a57937700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a57937700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a5015db30 0x7f1a500c13e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a57937700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5015b530 con 0x7f1a50142900 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a57937700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5015b810 con 0x7f1a5015db30 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a57937700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1a5015baf0 con 0x7f1a5006e960 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a54ed2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a54ed2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c3500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35926/0 (socket says 172.21.15.123:35926) 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.210+0000 7f1a54ed2700 1 -- 172.21.15.123:0/2557496902 learned_addr learned my addr 172.21.15.123:0/2557496902 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a556d3700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a5006e960 0x7f1a50157ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a5015db30 0x7f1a500c13e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a54ed2700 1 -- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a5006e960 msgr2=0x7f1a50157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a5006e960 0x7f1a50157ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a54ed2700 1 -- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a5015db30 msgr2=0x7f1a500c13e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a5015db30 0x7f1a500c13e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a54ed2700 1 -- 172.21.15.123:0/2557496902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1a4c009420 con 0x7f1a50142900 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a556d3700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a5006e960 0x7f1a50157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c3500 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000c510 tx=0x7f1a4000c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a457fa700 1 -- 172.21.15.123:0/2557496902 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40011590 con 0x7f1a50142900 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a457fa700 1 -- 172.21.15.123:0/2557496902 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a40011710 con 0x7f1a50142900 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1a5015bdd0 con 0x7f1a50142900 2023-10-06T04:32:30.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a457fa700 1 -- 172.21.15.123:0/2557496902 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40011ad0 con 0x7f1a50142900 2023-10-06T04:32:30.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.211+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a50068930 con 0x7f1a50142900 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.212+0000 7f1a457fa700 1 -- 172.21.15.123:0/2557496902 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1a4001b3b0 con 0x7f1a50142900 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.213+0000 7f1a457fa700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a34066f30 0x7f1a340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.213+0000 7f1a457fa700 1 -- 172.21.15.123:0/2557496902 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a40083bf0 con 0x7f1a50142900 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.213+0000 7f1a556d3700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a34066f30 0x7f1a340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.213+0000 7f1a556d3700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a34066f30 0x7f1a340693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c003080 tx=0x7f1a4c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.215+0000 7f1a60c4bb80 1 Processor -- start 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.215+0000 7f1a60c4bb80 1 -- start start 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.215+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174fa1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.215+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fa29f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 0x563174f9c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563174eba310 con 0x563174fa1770 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563174ef7130 con 0x563174f9c260 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x563174ef4af0 con 0x563174feb370 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174fa1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174fa1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35928/0 (socket says 172.21.15.123:35928) 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 -- 172.21.15.123:0/1694879943 learned_addr learned my addr 172.21.15.123:0/1694879943 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fa29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 -- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 msgr2=0x563174fa29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fa29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 -- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 msgr2=0x563174f9c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 0x563174f9c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 0x563174f9c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 -- 172.21.15.123:0/1694879943 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563174cf6790 con 0x563174fa1770 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fa29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a556d3700 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174fa1b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1a500713f0 tx=0x7f1a4c01e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ca8f4ed3e2ec164 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a3a7fc700 1 -- 172.21.15.123:0/1694879943 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c01cc30 con 0x563174fa1770 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a3a7fc700 1 -- 172.21.15.123:0/1694879943 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c01cd90 con 0x563174fa1770 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.216+0000 7f1a3a7fc700 1 -- 172.21.15.123:0/1694879943 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c01a640 con 0x563174fa1770 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/1694879943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 msgr2=0x563174fa1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174fa1b40 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f1a500713f0 tx=0x7f1a4c01e040 comp rx=0 tx=0).stop 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/1694879943 shutdown_connections 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174fa1b40 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 0x563174f9c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/1694879943 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fa29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/1694879943 >> 172.21.15.123:0/1694879943 conn(0x563174fea890 msgr2=0x563174feac60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/1694879943 shutdown_connections 2023-10-06T04:32:30.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/1694879943 wait complete. 2023-10-06T04:32:30.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.217+0000 7f1a60c4bb80 1 -- start start 2023-10-06T04:32:30.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 1 Processor -- start 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174f9c260 0x563174ff6480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174fa1770 0x563174fce520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fcea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563174ef08e0 con 0x563174f9c260 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563174e9c980 con 0x563174fa1770 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x563174e9c620 con 0x563174feb370 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a60c4bb80 10 client.0 _ll_register_callbacks cb 0x563174cffa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a556d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174f9c260 0x563174ff6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a556d3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174f9c260 0x563174ff6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35944/0 (socket says 172.21.15.123:35944) 2023-10-06T04:32:30.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.218+0000 7f1a556d3700 1 -- 172.21.15.123:0/3413522682 learned_addr learned my addr 172.21.15.123:0/3413522682 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fcea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174fa1770 0x563174fce520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a556d3700 1 -- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 msgr2=0x563174fcea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a556d3700 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fcea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a556d3700 1 -- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174fa1770 msgr2=0x563174fce520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a556d3700 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174fa1770 0x563174fce520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a556d3700 1 -- 172.21.15.123:0/3413522682 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a4c022070 con 0x563174f9c260 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a556d3700 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174f9c260 0x563174ff6480 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098e0 tx=0x7f1a4c02a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4087ef8ab6e06653 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/3413522682 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a4c036070 con 0x563174f9c260 2023-10-06T04:32:30.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/3413522682 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a4c03bc20 con 0x563174f9c260 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x563174f94e50 con 0x563174f9c260 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/3413522682 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4c01a640 con 0x563174f9c260 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 10 client.5223 fetch_fsmap learned FSMap version 17 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563174de30a0 con 0x563174f9c260 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/3413522682 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x563174f94e50 con 0x563174f9c260 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/3413522682 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1a4c02f020 con 0x563174f9c260 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 10 client.5223 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 10 client.5223 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:32:30.229 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 20 client.5223 populate_metadata read hostname 'smithi123' 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcd640 0x563174fcda10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 4 client.5223 mds_command: new command op to 5031 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.219+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 --> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x563174f71c20 con 0x563174fcd640 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.220+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcd640 0x563174fcda10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.220+0000 7f1a2f7fe700 10 client.5223 ms_handle_connect on v2:172.21.15.177:6833/1099021797 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.220+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcd640 0x563174fcda10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b87059234e19b924 server_cookie=cad2b567f3c00899 in_seq=0 out_seq=0 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/3413522682 <== mds.0 v2:172.21.15.177:6833/1099021797 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x563174f71c20 con 0x563174fcd640 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a2f7fe700 10 client.5223 handle_command_reply: tid=0 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 client.5223 shutdown 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174f9c260 msgr2=0x563174ff6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174f9c260 0x563174ff6480 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c0098e0 tx=0x7f1a4c02a000 comp rx=0 tx=0).stop 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 shutdown_connections 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcd640 0x563174fcda10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174f9c260 0x563174ff6480 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174feb370 0x563174fcea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/3413522682 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174fa1770 0x563174fce520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a2f7fe700 0 client.5223 ms_handle_reset on v2:172.21.15.177:6833/1099021797 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 >> 172.21.15.123:0/3413522682 conn(0x563174fea890 msgr2=0x563174fe7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 shutdown_connections 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.221+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/3413522682 wait complete. 2023-10-06T04:32:30.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.222+0000 7f1a60c4bb80 20 client.5223 trim_cache size 0 max 16384 2023-10-06T04:32:30.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.238+0000 7f1a60c4bb80 1 Processor -- start 2023-10-06T04:32:30.240 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.238+0000 7f1a60c4bb80 1 -- start start 2023-10-06T04:32:30.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 0x563174fc98e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174fa1770 0x563174feac60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174feb370 0x563174fca0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x563174d881e0 con 0x563174feb370 2023-10-06T04:32:30.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x563174fca630 con 0x563174f9c260 2023-10-06T04:32:30.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x563174f97340 con 0x563174fa1770 2023-10-06T04:32:30.243 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174feb370 0x563174fca0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174feb370 0x563174fca0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35954/0 (socket says 172.21.15.123:35954) 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 -- 172.21.15.123:0/2999198699 learned_addr learned my addr 172.21.15.123:0/2999198699 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 -- 172.21.15.123:0/2999198699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174fa1770 msgr2=0x563174feac60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/2999198699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174fa1770 0x563174feac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 -- 172.21.15.123:0/2999198699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 msgr2=0x563174fc98e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/2999198699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 0x563174fc98e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 -- 172.21.15.123:0/2999198699 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x563174fcb860 con 0x563174feb370 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/2999198699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174feb370 0x563174fca0f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x563174fcb9f0 tx=0x7f1a48009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24bb86ec14406d32 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a3affd700 1 -- 172.21.15.123:0/2999198699 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48019030 con 0x563174feb370 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a3affd700 1 -- 172.21.15.123:0/2999198699 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4800d040 con 0x563174feb370 2023-10-06T04:32:30.244 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.239+0000 7f1a3affd700 1 -- 172.21.15.123:0/2999198699 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a48011770 con 0x563174feb370 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/2999198699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174feb370 msgr2=0x563174fca0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/2999198699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174feb370 0x563174fca0f0 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x563174fcb9f0 tx=0x7f1a48009040 comp rx=0 tx=0).stop 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/2999198699 shutdown_connections 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/2999198699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174feb370 0x563174fca0f0 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/2999198699 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174f9c260 0x563174fc98e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/2999198699 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174fa1770 0x563174feac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/2999198699 >> 172.21.15.123:0/2999198699 conn(0x563174fea890 msgr2=0x563174fe7f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/2999198699 shutdown_connections 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.240+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/2999198699 wait complete. 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 -- start start 2023-10-06T04:32:30.245 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 Processor -- start 2023-10-06T04:32:30.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174f9c260 0x563174f9c800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174f9cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174feb370 0x563174fcc200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317504c7d0 con 0x563174fa1770 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x56317504cf30 con 0x563174feb370 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.241+0000 7f1a60c4bb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x56317504bce0 con 0x563174f9c260 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a60c4bb80 10 client.0 _ll_register_callbacks cb 0x563174cffa00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174f9cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174f9cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:35964/0 (socket says 172.21.15.123:35964) 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 -- 172.21.15.123:0/538734035 learned_addr learned my addr 172.21.15.123:0/538734035 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:30.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a556d3700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174f9c260 0x563174f9c800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174feb370 0x563174fcc200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 -- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174f9c260 msgr2=0x563174f9c800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174f9c260 0x563174f9c800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 -- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174feb370 msgr2=0x563174fcc200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174feb370 0x563174fcc200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 -- 172.21.15.123:0/538734035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1a3c003570 con 0x563174fa1770 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a55ed4700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174feb370 0x563174fcc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a54ed2700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174f9cd40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1a40054670 tx=0x7f1a40081f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d500bdd9f360d9fe server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a2f7fe700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/538734035 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1a40052d60 con 0x563174fa1770 2023-10-06T04:32:30.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/538734035 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1a40053eb0 con 0x563174fa1770 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f1a4c01cd90 con 0x563174fa1770 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/538734035 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1a4005b4d0 con 0x563174fa1770 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a60c4bb80 10 client.5229 fetch_fsmap learned FSMap version 17 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=2+}) v3 -- 0x563174fd0780 con 0x563174fa1770 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/538734035 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1a4c01cd90 con 0x563174fa1770 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/538734035 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 17) v1 ==== 3100+0+0 (secure 0 0 0) 0x7f1a400702c0 con 0x563174fa1770 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a60c4bb80 10 client.5229 fetch_fsmap finished waiting for FSMap version 17 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a60c4bb80 10 client.5229 resolve_mds: resolved name 'd' to daemon mds.d 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.242+0000 7f1a60c4bb80 20 client.5229 populate_metadata read hostname 'smithi123' 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcc740 0x563174fffa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 4 client.5229 mds_command: new command op to 5031 tid=0[{"prefix": "status"}] 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 --> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] -- command(tid 0: {"prefix": "status"}) v1 -- 0x563174e938d0 con 0x563174fcc740 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a556d3700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcc740 0x563174fffa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a2f7fe700 10 client.5229 ms_handle_connect on v2:172.21.15.177:6833/1099021797 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a556d3700 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcc740 0x563174fffa40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75c8eb125c602be9 server_cookie=28ad189955a0143f in_seq=0 out_seq=0 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a2f7fe700 1 -- 172.21.15.123:0/538734035 <== mds.0 v2:172.21.15.177:6833/1099021797 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+318 (crc 0 0 0) 0x563174e938d0 con 0x563174fcc740 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a2f7fe700 10 client.5229 handle_command_reply: tid=0 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 client.5229 shutdown 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 msgr2=0x563174f9cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174f9cd40 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f1a40054670 tx=0x7f1a40081f10 comp rx=0 tx=0).stop 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 shutdown_connections 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:6833/1099021797,v1:172.21.15.177:6835/1099021797] conn(0x563174fcc740 0x563174fffa40 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x563174feb370 0x563174fcc200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x563174f9c260 0x563174f9c800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 --2- 172.21.15.123:0/538734035 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x563174fa1770 0x563174f9cd40 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a2f7fe700 0 client.5229 ms_handle_reset on v2:172.21.15.177:6833/1099021797 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.243+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 >> 172.21.15.123:0/538734035 conn(0x563174fea890 msgr2=0x563174ff60b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 shutdown_connections 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a60c4bb80 1 -- 172.21.15.123:0/538734035 wait complete. 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a60c4bb80 20 client.5229 trim_cache size 0 max 16384 2023-10-06T04:32:30.254 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "1cea82d3-d2ef-4e20-8c3e-67296f7df92c", 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 5031, 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs", 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 5.815431244, 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 17, 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 34, 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 34, 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 13.688649302 2023-10-06T04:32:30.255 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:30.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a34066f30 msgr2=0x7f1a340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a34066f30 0x7f1a340693e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f1a4c003080 tx=0x7f1a4c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 msgr2=0x7f1a500c3500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c3500 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7f1a4000c510 tx=0x7f1a4000c8d0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 shutdown_connections 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a34066f30 0x7f1a340693e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1a5015db30 0x7f1a500c13e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1a5006e960 0x7f1a50157ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 --2- 172.21.15.123:0/2557496902 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1a50142900 0x7f1a500c3500 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 >> 172.21.15.123:0/2557496902 conn(0x7f1a500cd010 msgr2=0x7f1a5015cd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 shutdown_connections 2023-10-06T04:32:30.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.244+0000 7f1a57937700 1 -- 172.21.15.123:0/2557496902 wait complete. 2023-10-06T04:32:30.259 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:32:30.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.373+0000 7f1e41095700 1 Processor -- start 2023-10-06T04:32:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.373+0000 7f1e41095700 1 -- start start 2023-10-06T04:32:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.374+0000 7f1e41095700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0aae60 0x7f1e3c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.374+0000 7f1e41095700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1e3c0ab230 0x7f1e3c0afbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.374+0000 7f1e41095700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e3c0abe30 con 0x7f1e3c0ace40 2023-10-06T04:32:30.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.374+0000 7f1e41095700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e3c0ac9b0 con 0x7f1e3c0ab230 2023-10-06T04:32:30.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.374+0000 7f1e41095700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0ac080 con 0x7f1e3c0aae60 2023-10-06T04:32:30.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.374+0000 7f1e3b59e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1e3c0ace40 0x7f1e3c068770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33184/0 (socket says 172.21.15.123:33184) 2023-10-06T04:32:30.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.374+0000 7f1e3b59e700 1 -- 172.21.15.123:0/3637865539 learned_addr learned my addr 172.21.15.123:0/3637865539 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:30.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e3ad9d700 1 --2- 172.21.15.123:0/3637865539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0aae60 0x7f1e3c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e3a59c700 1 --2- 172.21.15.123:0/3637865539 >> v2:172.21.15.177:3300/0 conn(0x7f1e3c0ab230 0x7f1e3c0afbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.380 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e39d9b700 1 -- 172.21.15.123:0/3637865539 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2082425901 0 0) 0x7f1e3c0abe30 con 0x7f1e3c0ace40 2023-10-06T04:32:30.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e39d9b700 1 -- 172.21.15.123:0/3637865539 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e24003570 con 0x7f1e3c0ace40 2023-10-06T04:32:30.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e3ad9d700 1 -- 172.21.15.123:0/3637865539 >> v2:172.21.15.177:3300/0 conn(0x7f1e3c0ab230 msgr2=0x7f1e3c0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:30.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e3ad9d700 1 --2- 172.21.15.123:0/3637865539 >> v2:172.21.15.177:3300/0 conn(0x7f1e3c0ab230 0x7f1e3c0afbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e3ad9d700 1 -- 172.21.15.123:0/3637865539 >> v1:172.21.15.123:6789/0 conn(0x7f1e3c0ace40 legacy=0x7f1e3c068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e3ad9d700 1 -- 172.21.15.123:0/3637865539 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e3c0ac360 con 0x7f1e3c0aae60 2023-10-06T04:32:30.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e3ad9d700 1 --2- 172.21.15.123:0/3637865539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0aae60 0x7f1e3c0aec10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0098e0 tx=0x7f1e2c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e90243be03559011 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e39d9b700 1 -- 172.21.15.123:0/3637865539 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c011630 con 0x7f1e3c0aae60 2023-10-06T04:32:30.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e39d9b700 1 -- 172.21.15.123:0/3637865539 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e2c011790 con 0x7f1e3c0aae60 2023-10-06T04:32:30.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.375+0000 7f1e39d9b700 1 -- 172.21.15.123:0/3637865539 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e2c011b10 con 0x7f1e3c0aae60 2023-10-06T04:32:30.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 -- 172.21.15.123:0/3637865539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0aae60 msgr2=0x7f1e3c0aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 --2- 172.21.15.123:0/3637865539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0aae60 0x7f1e3c0aec10 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c0098e0 tx=0x7f1e2c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 -- 172.21.15.123:0/3637865539 shutdown_connections 2023-10-06T04:32:30.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 --2- 172.21.15.123:0/3637865539 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0aae60 0x7f1e3c0aec10 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 --2- 172.21.15.123:0/3637865539 >> v2:172.21.15.177:3300/0 conn(0x7f1e3c0ab230 0x7f1e3c0afbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 -- 172.21.15.123:0/3637865539 >> 172.21.15.123:0/3637865539 conn(0x7f1e3c097410 msgr2=0x7f1e3c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 -- 172.21.15.123:0/3637865539 shutdown_connections 2023-10-06T04:32:30.383 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.376+0000 7f1e41095700 1 -- 172.21.15.123:0/3637865539 wait complete. 2023-10-06T04:32:30.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.377+0000 7f1e41095700 1 Processor -- start 2023-10-06T04:32:30.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.377+0000 7f1e41095700 1 -- start start 2023-10-06T04:32:30.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e41095700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e3c0aae60 0x7f1e3c0b3740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e41095700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c0ab230 0x7f1e3c171a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e41095700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0ace40 0x7f1e3c151b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e41095700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c16e600 con 0x7f1e3c0ab230 2023-10-06T04:32:30.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e41095700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b1dc0 con 0x7f1e3c0aae60 2023-10-06T04:32:30.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e41095700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3c0b20a0 con 0x7f1e3c0ace40 2023-10-06T04:32:30.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c0ab230 0x7f1e3c171a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.387 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c0ab230 0x7f1e3c171a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48060/0 (socket says 172.21.15.123:48060) 2023-10-06T04:32:30.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 -- 172.21.15.123:0/3454480982 learned_addr learned my addr 172.21.15.123:0/3454480982 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:30.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3ad9d700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e3c0aae60 0x7f1e3c0b3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3b59e700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0ace40 0x7f1e3c151b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 -- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0ace40 msgr2=0x7f1e3c151b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0ace40 0x7f1e3c151b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 -- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e3c0aae60 msgr2=0x7f1e3c0b3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.388 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e3c0aae60 0x7f1e3c0b3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.378+0000 7f1e3a59c700 1 -- 172.21.15.123:0/3454480982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e2c009420 con 0x7f1e3c0ab230 2023-10-06T04:32:30.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.379+0000 7f1e3a59c700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c0ab230 0x7f1e3c171a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000ec80 tx=0x7f1e300091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.379+0000 7f1e22ffd700 1 -- 172.21.15.123:0/3454480982 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3000c9d0 con 0x7f1e3c0ab230 2023-10-06T04:32:30.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.379+0000 7f1e22ffd700 1 -- 172.21.15.123:0/3454480982 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3000cb30 con 0x7f1e3c0ab230 2023-10-06T04:32:30.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.379+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3c0b2380 con 0x7f1e3c0ab230 2023-10-06T04:32:30.389 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.379+0000 7f1e22ffd700 1 -- 172.21.15.123:0/3454480982 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e30013470 con 0x7f1e3c0ab230 2023-10-06T04:32:30.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.379+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3c0b3c80 con 0x7f1e3c0ab230 2023-10-06T04:32:30.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.380+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3c16e410 con 0x7f1e3c0ab230 2023-10-06T04:32:30.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.380+0000 7f1e22ffd700 1 -- 172.21.15.123:0/3454480982 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e30013820 con 0x7f1e3c0ab230 2023-10-06T04:32:30.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.380+0000 7f1e22ffd700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e28066f30 0x7f1e280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:30.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.381+0000 7f1e22ffd700 1 -- 172.21.15.123:0/3454480982 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7f1e30084a80 con 0x7f1e3c0ab230 2023-10-06T04:32:30.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.381+0000 7f1e3ad9d700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e28066f30 0x7f1e280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:30.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.381+0000 7f1e3ad9d700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e28066f30 0x7f1e280693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c006dc0 tx=0x7f1e2c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:30.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.385+0000 7f1e22ffd700 1 -- 172.21.15.123:0/3454480982 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e3c16e410 con 0x7f1e3c0ab230 2023-10-06T04:32:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.609+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1e3c16e410 con 0x7f1e3c0ab230 2023-10-06T04:32:30.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.609+0000 7f1e22ffd700 1 -- 172.21.15.123:0/3454480982 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1e3c16e410 con 0x7f1e3c0ab230 2023-10-06T04:32:30.612 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:32:30.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e28066f30 msgr2=0x7f1e280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e28066f30 0x7f1e280693e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f1e2c006dc0 tx=0x7f1e2c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c0ab230 msgr2=0x7f1e3c171a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c0ab230 0x7f1e3c171a20 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f1e3000ec80 tx=0x7f1e300091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 shutdown_connections 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e28066f30 0x7f1e280693e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e3c0ace40 0x7f1e3c151b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e3c0aae60 0x7f1e3c0b3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 --2- 172.21.15.123:0/3454480982 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e3c0ab230 0x7f1e3c171a20 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 >> 172.21.15.123:0/3454480982 conn(0x7f1e3c097410 msgr2=0x7f1e3c0cea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 shutdown_connections 2023-10-06T04:32:30.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:30.611+0000 7f1e41095700 1 -- 172.21.15.123:0/3454480982 wait complete. 2023-10-06T04:32:30.625 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-10-06T04:32:30.625 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:32:30.625 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-10-06T04:32:30.625 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:32:30.625 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:30.626 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-10-06T04:32:30.626 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:32:30.626 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:32:30.626 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:32:30.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:30.626 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:30.645 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:32:30.646 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42464sec preferred_lft 42464sec 2023-10-06T04:32:30.647 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2023-10-06T04:32:30.647 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:30.647 INFO:teuthology.orchestra.run.smithi123.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:32:30.647 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:ca:fa:1b:04:98 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:30.647 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5073:dcff:fe45:d49e/64 scope link 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:e0:39:ac:6a:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::44e0:39ff:feac:6a27/64 scope link 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:ca:fa:1b:04:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::44ca:faff:fe1b:498/64 scope link 2023-10-06T04:32:30.648 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:30.649 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:30.649 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:32:30.711 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-10-06T04:32:30.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:30.712 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-10-06T04:32:30.772 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-10-06T04:32:30.773 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:32:30.773 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:32:30.773 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:32:30.773 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:32:30.773 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:30.773 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:32:30.773 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:32:30.773 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:32:30.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:32:30.774 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:32:30.774 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:32:30.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:30.878 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-10-06T04:32:30.895 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:32:30.896 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:32:30.896 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:32:30.896 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:30.896 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2023-10-06T04:32:30.965 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:32:30.966 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:32:30.966 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:31.023 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:32:31.023 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:32:31.024 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:31.081 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:32:31.148 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:32:31.149 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:32:31.215 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:32:31.216 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:32:31.216 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:31.217 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:32:31.275 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:32:31.275 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0) 2023-10-06T04:32:31.318 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:32:31.345 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:32:31.345 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:32:31.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:2023-10-06T04:32:31.398+0000 7ff2ce1b0540 -1 init, newargv = 0x56258b5f3840 newargc=15 2023-10-06T04:32:31.400 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[134537]: starting ceph client 2023-10-06T04:32:31.406 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[134537]: starting fuse 2023-10-06T04:32:31.412 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:32:31.413 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:32:31.413 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:31.414 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:32:31.472 INFO:teuthology.orchestra.run.smithi123.stdout:46 2023-10-06T04:32:31.472 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:32:31.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:32:31.472 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> import re 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2023-10-06T04:32:31.473 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> return f 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.0")) 2023-10-06T04:32:31.474 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-10-06T04:32:31.595 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.0.134537.asok 2023-10-06T04:32:31.778 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.134537.asok 2023-10-06T04:32:31.778 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:31.778 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.134537.asok status 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "0", 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "134537", 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2023-10-06T04:32:31.931 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 5244, 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 5244 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:32:31.932 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1455482126 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 1455482126 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.5244 192.168.0.1:0/1455482126", 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/1455482126", 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2023-10-06T04:32:31.933 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 17, 2023-10-06T04:32:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 34, 2023-10-06T04:32:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:32:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2023-10-06T04:32:31.934 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "cephfs" 2023-10-06T04:32:31.934 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:31.940 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:31.941 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2023-10-06T04:32:31.941 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:31.959 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2023-10-06T04:32:31.959 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:31.961 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:31.961 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:32.027 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:32.027 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir subdir) 2023-10-06T04:32:32.093 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:32.093 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec dd if=/dev/urandom of=subdir/sixmegs bs=1M conv=fdatasync count=6 seek=0) 2023-10-06T04:32:32.211 INFO:teuthology.orchestra.run.smithi123.stderr:6+0 records in 2023-10-06T04:32:32.211 INFO:teuthology.orchestra.run.smithi123.stderr:6+0 records out 2023-10-06T04:32:32.211 INFO:teuthology.orchestra.run.smithi123.stderr:6291456 bytes (6.3 MB, 6.0 MiB) copied, 0.0574382 s, 110 MB/s 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> import stat 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> import json 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> import sys 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> try: 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> except OSError as e: 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> sys.exit(e.errno) 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:32.213 DEBUG:teuthology.orchestra.run.smithi123:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T04:32:32.214 DEBUG:teuthology.orchestra.run.smithi123:> print(json.dumps( 2023-10-06T04:32:32.214 DEBUG:teuthology.orchestra.run.smithi123:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T04:32:32.214 DEBUG:teuthology.orchestra.run.smithi123:> indent=2)) 2023-10-06T04:32:32.214 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-10-06T04:32:32.287 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:32.287 INFO:teuthology.orchestra.run.smithi123.stdout: "st_mode": 33204, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_ino": 1099511627777, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_dev": 46, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_nlink": 1, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_uid": 1000, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_gid": 1285, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_size": 6291456, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_atime": 1696566752.1518312, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_mtime": 1696566752.1913884, 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout: "st_ctime": 1696566752.1913884 2023-10-06T04:32:32.288 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:32.471 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2023-10-06T04:32:32.471 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:32:32.472 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:32.554 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[134537]: fuse finished with error 0 and tester_r 0 2023-10-06T04:32:32.556 INFO:teuthology.orchestra.run:waiting for 900 2023-10-06T04:32:32.595 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:32:32.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:32.595 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:32:32.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:32.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.809+0000 7ff59aae4700 1 Processor -- start 2023-10-06T04:32:32.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.809+0000 7ff59aae4700 1 -- start start 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff59aae4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff59406cd60 0x7ff5940ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff59aae4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff594157910 con 0x7ff594157330 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff59aae4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff59406d240 con 0x7ff594156f60 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff59aae4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff59406d520 con 0x7ff59406cd60 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff593fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff59406cd60 0x7ff5940ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff599081700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff594157330 0x7ff594157da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33194/0 (socket says 172.21.15.123:33194) 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff599081700 1 -- 172.21.15.123:0/287351157 learned_addr learned my addr 172.21.15.123:0/287351157 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff593fff700 1 -- 172.21.15.123:0/287351157 >> v1:172.21.15.177:6789/0 conn(0x7ff594156f60 legacy=0x7ff59415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff593fff700 1 -- 172.21.15.123:0/287351157 >> v1:172.21.15.123:6789/0 conn(0x7ff594157330 legacy=0x7ff594157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:32.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff593fff700 1 -- 172.21.15.123:0/287351157 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff59406d800 con 0x7ff59406cd60 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff5937fe700 1 -- 172.21.15.123:0/287351157 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3771664602 0 0) 0x7ff594157910 con 0x7ff594157330 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.810+0000 7ff593fff700 1 --2- 172.21.15.123:0/287351157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59406cd60 0x7ff5940ad730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff580009910 tx=0x7ff5800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d373f2ad74e440 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff5937fe700 1 -- 172.21.15.123:0/287351157 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000b7e0 con 0x7ff59406cd60 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff5937fe700 1 -- 172.21.15.123:0/287351157 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff58000b940 con 0x7ff59406cd60 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff5937fe700 1 -- 172.21.15.123:0/287351157 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58000bce0 con 0x7ff59406cd60 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff59aae4700 1 -- 172.21.15.123:0/287351157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59406cd60 msgr2=0x7ff5940ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff59aae4700 1 --2- 172.21.15.123:0/287351157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59406cd60 0x7ff5940ad730 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7ff580009910 tx=0x7ff5800091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff59aae4700 1 -- 172.21.15.123:0/287351157 shutdown_connections 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff59aae4700 1 --2- 172.21.15.123:0/287351157 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff59406cd60 0x7ff5940ad730 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff59aae4700 1 -- 172.21.15.123:0/287351157 >> 172.21.15.123:0/287351157 conn(0x7ff5940ce0c0 msgr2=0x7ff5940ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff59aae4700 1 -- 172.21.15.123:0/287351157 shutdown_connections 2023-10-06T04:32:32.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.811+0000 7ff59aae4700 1 -- 172.21.15.123:0/287351157 wait complete. 2023-10-06T04:32:32.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.812+0000 7ff59aae4700 1 Processor -- start 2023-10-06T04:32:32.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.812+0000 7ff59aae4700 1 -- start start 2023-10-06T04:32:32.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff59aae4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff59406cd60 0x7ff594168ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff59aae4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff594156f60 0x7ff5940c2640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff59aae4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff594157330 0x7ff5940c94e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff59aae4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff59415a170 con 0x7ff594156f60 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff59aae4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff594068810 con 0x7ff594157330 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff59aae4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff594068af0 con 0x7ff59406cd60 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff593fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff594156f60 0x7ff5940c2640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff593fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff594156f60 0x7ff5940c2640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48086/0 (socket says 172.21.15.123:48086) 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.813+0000 7ff593fff700 1 -- 172.21.15.123:0/1696670930 learned_addr learned my addr 172.21.15.123:0/1696670930 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff593fff700 1 -- 172.21.15.123:0/1696670930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff59406cd60 msgr2=0x7ff594168ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:32.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff599081700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff594157330 0x7ff5940c94e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff593fff700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff59406cd60 0x7ff594168ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff593fff700 1 -- 172.21.15.123:0/1696670930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff594157330 msgr2=0x7ff5940c94e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff593fff700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff594157330 0x7ff5940c94e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff593fff700 1 -- 172.21.15.123:0/1696670930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580009420 con 0x7ff594156f60 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff593fff700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff594156f60 0x7ff5940c2640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5800093f0 tx=0x7ff580016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff590ff9700 1 -- 172.21.15.123:0/1696670930 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58001b450 con 0x7ff594156f60 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff590ff9700 1 -- 172.21.15.123:0/1696670930 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff580020070 con 0x7ff594156f60 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff594068dd0 con 0x7ff594156f60 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff590ff9700 1 -- 172.21.15.123:0/1696670930 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff58001b820 con 0x7ff594156f60 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.814+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff594169530 con 0x7ff594156f60 2023-10-06T04:32:32.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.815+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff594069270 con 0x7ff594156f60 2023-10-06T04:32:32.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.815+0000 7ff590ff9700 1 -- 172.21.15.123:0/1696670930 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff580022030 con 0x7ff594156f60 2023-10-06T04:32:32.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.816+0000 7ff590ff9700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:32.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.816+0000 7ff590ff9700 1 -- 172.21.15.123:0/1696670930 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7ff580086720 con 0x7ff594156f60 2023-10-06T04:32:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.816+0000 7ff598880700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:32.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.816+0000 7ff598880700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff578066f30 0x7ff5780693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff588007fd0 tx=0x7ff588007b20 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:32.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:32.820+0000 7ff590ff9700 1 -- 172.21.15.123:0/1696670930 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff580053830 con 0x7ff594156f60 2023-10-06T04:32:33.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.045+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff594069270 con 0x7ff594156f60 2023-10-06T04:32:33.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.046+0000 7ff590ff9700 1 -- 172.21.15.123:0/1696670930 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) v1 ==== 94+0+4609 (secure 0 0 0) 0x7ff580056e50 con 0x7ff594156f60 2023-10-06T04:32:33.048 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.049 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":17,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4909,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6834/2836433084","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":2836433084},{"type":"v1","addr":"172.21.15.177:6834","nonce":2836433084}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12},{"gid":4912,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/3003404474","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":3003404474},{"type":"v1","addr":"172.21.15.123:6836","nonce":3003404474}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5028,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/1646131544","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":1646131544},{"type":"v1","addr":"172.21.15.123:6837","nonce":1646131544}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":12}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:27.453433+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"d","rank":0,"incarnation":13,"state":"up:active","state_seq":3,"addr":"172.21.15.177:6835/1099021797","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6833","nonce":1099021797},{"type":"v1","addr":"172.21.15.177:6835","nonce":1099021797}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2}]} 2023-10-06T04:32:33.050 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff578066f30 msgr2=0x7ff5780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff578066f30 0x7ff5780693e0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7ff588007fd0 tx=0x7ff588007b20 comp rx=0 tx=0).stop 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff594156f60 msgr2=0x7ff5940c2640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff594156f60 0x7ff5940c2640 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7ff5800093f0 tx=0x7ff580016000 comp rx=0 tx=0).stop 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 shutdown_connections 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff578066f30 0x7ff5780693e0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff594157330 0x7ff5940c94e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff59406cd60 0x7ff594168ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 --2- 172.21.15.123:0/1696670930 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff594156f60 0x7ff5940c2640 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 >> 172.21.15.123:0/1696670930 conn(0x7ff5940ce0c0 msgr2=0x7ff5940af680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 shutdown_connections 2023-10-06T04:32:33.051 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.048+0000 7ff59aae4700 1 -- 172.21.15.123:0/1696670930 wait complete. 2023-10-06T04:32:33.052 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 17 2023-10-06T04:32:33.062 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph --admin-daemon /var/run/ceph/ceph-mds.d.asok --format=json flush journal 2023-10-06T04:32:33.221 INFO:teuthology.orchestra.run.smithi177.stdout:{"message":"","return_code":0} 2023-10-06T04:32:33.221 DEBUG:tasks.cephfs.filesystem:_json_asok output { "message": "", "return_code": 0 } 2023-10-06T04:32:33.221 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:32:33.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.339+0000 7fad591ca700 1 Processor -- start 2023-10-06T04:32:33.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.339+0000 7fad591ca700 1 -- start start 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad591ca700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fad540b0050 0x7fad540c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad591ca700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fad5406bb80 0x7fad5415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad591ca700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad5415cfa0 con 0x7fad540b0420 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad591ca700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad540cedc0 con 0x7fad5406bb80 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad591ca700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad54069780 con 0x7fad540b0050 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fad5406bb80 0x7fad5415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fad5406bb80 0x7fad5415a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48090/0 (socket says 172.21.15.123:48090) 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 -- 172.21.15.123:0/1138868699 learned_addr learned my addr 172.21.15.123:0/1138868699 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad52d9d700 1 --2- 172.21.15.123:0/1138868699 >> v2:172.21.15.177:3300/0 conn(0x7fad540b0050 0x7fad540c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 -- 172.21.15.123:0/1138868699 >> v2:172.21.15.177:3300/0 conn(0x7fad540b0050 msgr2=0x7fad540c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:33.344 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 --2- 172.21.15.123:0/1138868699 >> v2:172.21.15.177:3300/0 conn(0x7fad540b0050 0x7fad540c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 -- 172.21.15.123:0/1138868699 >> v1:172.21.15.177:6789/0 conn(0x7fad540b0420 legacy=0x7fad5406b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 -- 172.21.15.123:0/1138868699 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad5415a910 con 0x7fad5406bb80 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad52d9d700 1 --2- 172.21.15.123:0/1138868699 >> v2:172.21.15.177:3300/0 conn(0x7fad540b0050 0x7fad540c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad5359e700 1 --2- 172.21.15.123:0/1138868699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad5406bb80 0x7fad5415a3d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad5406c140 tx=0x7fad4c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515080a13318eb8b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad51d9b700 1 -- 172.21.15.123:0/1138868699 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c00e070 con 0x7fad5406bb80 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad51d9b700 1 -- 172.21.15.123:0/1138868699 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c00ba10 con 0x7fad5406bb80 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.340+0000 7fad51d9b700 1 -- 172.21.15.123:0/1138868699 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0092e0 con 0x7fad5406bb80 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 -- 172.21.15.123:0/1138868699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad5406bb80 msgr2=0x7fad5415a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 --2- 172.21.15.123:0/1138868699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad5406bb80 0x7fad5415a3d0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fad5406c140 tx=0x7fad4c009840 comp rx=0 tx=0).stop 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 -- 172.21.15.123:0/1138868699 shutdown_connections 2023-10-06T04:32:33.345 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 --2- 172.21.15.123:0/1138868699 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad5406bb80 0x7fad5415a3d0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 --2- 172.21.15.123:0/1138868699 >> v2:172.21.15.177:3300/0 conn(0x7fad540b0050 0x7fad540c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 -- 172.21.15.123:0/1138868699 >> 172.21.15.123:0/1138868699 conn(0x7fad54068130 msgr2=0x7fad540ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:33.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 -- 172.21.15.123:0/1138868699 shutdown_connections 2023-10-06T04:32:33.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.341+0000 7fad591ca700 1 -- 172.21.15.123:0/1138868699 wait complete. 2023-10-06T04:32:33.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.342+0000 7fad591ca700 1 Processor -- start 2023-10-06T04:32:33.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.342+0000 7fad591ca700 1 -- start start 2023-10-06T04:32:33.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad591ca700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad5406bb80 0x7fad540aeef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad591ca700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad540b0050 0x7fad540c24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad591ca700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad540b0420 0x7fad5406d290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad591ca700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad540af430 con 0x7fad540b0050 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad591ca700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad540c2dc0 con 0x7fad540b0420 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad591ca700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad54169970 con 0x7fad5406bb80 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad540b0050 0x7fad540c24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5359e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad540b0420 0x7fad5406d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad52d9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad5406bb80 0x7fad540aeef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad540b0050 0x7fad540c24e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48104/0 (socket says 172.21.15.123:48104) 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 -- 172.21.15.123:0/3169442809 learned_addr learned my addr 172.21.15.123:0/3169442809 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:33.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 -- 172.21.15.123:0/3169442809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad5406bb80 msgr2=0x7fad540aeef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad5406bb80 0x7fad540aeef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 -- 172.21.15.123:0/3169442809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad540b0420 msgr2=0x7fad5406d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad540b0420 0x7fad5406d290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.343+0000 7fad5259c700 1 -- 172.21.15.123:0/3169442809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c009ae0 con 0x7fad540b0050 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.344+0000 7fad5259c700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad540b0050 0x7fad540c24e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c5d0 tx=0x7fad44007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.344+0000 7fad3affd700 1 -- 172.21.15.123:0/3169442809 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44011590 con 0x7fad540b0050 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.344+0000 7fad3affd700 1 -- 172.21.15.123:0/3169442809 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad44011710 con 0x7fad540b0050 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.344+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad54169c50 con 0x7fad540b0050 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.344+0000 7fad3affd700 1 -- 172.21.15.123:0/3169442809 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad44011ad0 con 0x7fad540b0050 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.344+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad54169670 con 0x7fad540b0050 2023-10-06T04:32:33.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.345+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5416a170 con 0x7fad540b0050 2023-10-06T04:32:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.345+0000 7fad3affd700 1 -- 172.21.15.123:0/3169442809 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fad4401c460 con 0x7fad540b0050 2023-10-06T04:32:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.345+0000 7fad3affd700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fad40066f30 0x7fad400693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.346+0000 7fad52d9d700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fad40066f30 0x7fad400693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.346+0000 7fad3affd700 1 -- 172.21.15.123:0/3169442809 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 6058+0+0 (secure 0 0 0) 0x7fad44084350 con 0x7fad540b0050 2023-10-06T04:32:33.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.346+0000 7fad52d9d700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fad40066f30 0x7fad400693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad540b3840 tx=0x7fad48008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.352 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.350+0000 7fad3affd700 1 -- 172.21.15.123:0/3169442809 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fad5416a170 con 0x7fad540b0050 2023-10-06T04:32:33.480 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:33.482 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:32:33.580 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.577+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fad5416a170 con 0x7fad540b0050 2023-10-06T04:32:33.580 INFO:tasks.ceph.mon.a.smithi123.stderr:2023-10-06T04:32:33.578+0000 7fea90960700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:32:33.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.582+0000 7fad3affd700 1 -- 172.21.15.123:0/3169442809 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v18) v1 ==== 161+0+0 (secure 0 0 0) 0x7fad5416a170 con 0x7fad540b0050 2023-10-06T04:32:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fad40066f30 msgr2=0x7fad400693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fad40066f30 0x7fad400693e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fad540b3840 tx=0x7fad48008040 comp rx=0 tx=0).stop 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad540b0050 msgr2=0x7fad540c24e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad540b0050 0x7fad540c24e0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fad4400c5d0 tx=0x7fad44007f60 comp rx=0 tx=0).stop 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 shutdown_connections 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fad40066f30 0x7fad400693e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fad540b0420 0x7fad5406d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fad5406bb80 0x7fad540aeef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 --2- 172.21.15.123:0/3169442809 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fad540b0050 0x7fad540c24e0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 >> 172.21.15.123:0/3169442809 conn(0x7fad54068130 msgr2=0x7fad540ce0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:33.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 shutdown_connections 2023-10-06T04:32:33.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.584+0000 7fad591ca700 1 -- 172.21.15.123:0/3169442809 wait complete. 2023-10-06T04:32:33.588 INFO:tasks.ceph.mds.d.smithi177.stderr: -2928> 2023-10-06T04:32:28.412+0000 7fb1edb68700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:32:33.588 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:32:33.599 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:32:33.601 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:32:33.610 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-10-06T04:32:33.645 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2023-10-06T04:32:33.645 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2023-10-06T04:32:33.645 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:32:33.645 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2023-10-06T04:32:33.645 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.645 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2023-10-06T04:32:33.646 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2023-10-06T04:32:33.647 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2023-10-06T04:32:33.648 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2023-10-06T04:32:33.649 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2023-10-06T04:32:33.649 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.123,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T04:32:33.649 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:32:33.649 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.649 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2023-10-06T04:32:33.649 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2023-10-06T04:32:33.649 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2023-10-06T04:32:33.650 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2023-10-06T04:32:33.651 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: bdev async discard = True 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: bdev enable discard = True 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore allocator = bitmap 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 1/20 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 1/20 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2023-10-06T04:32:33.652 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 4/10 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.653 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2023-10-06T04:32:33.654 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:32:33.655 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2023-10-06T04:32:33.656 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.657 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-10-06T04:32:33.658 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-10-06T04:32:33.659 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2023-10-06T04:32:33.659 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2023-10-06T04:32:33.659 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.659 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2023-10-06T04:32:33.659 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = True 2023-10-06T04:32:33.660 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2023-10-06T04:32:33.660 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = False 2023-10-06T04:32:33.660 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-10-06T04:32:33.661 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.661 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.663 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2023-10-06T04:32:33.663 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2023-10-06T04:32:33.663 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:32:33.664 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2023-10-06T04:32:33.664 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.664 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2023-10-06T04:32:33.664 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.664 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2023-10-06T04:32:33.665 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.665 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2023-10-06T04:32:33.665 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2023-10-06T04:32:33.665 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.666 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2023-10-06T04:32:33.666 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2023-10-06T04:32:33.666 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.666 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:32:33.666 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2023-10-06T04:32:33.666 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.666 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:32:33.667 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.668 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.123,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:32:33.669 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.670 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2023-10-06T04:32:33.671 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:32:33.672 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.673 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2023-10-06T04:32:33.674 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.675 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2023-10-06T04:32:33.675 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2023-10-06T04:32:33.675 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.675 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.675 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:32:33.675 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:32:33.675 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2023-10-06T04:32:33.676 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2023-10-06T04:32:33.677 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.678 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2023-10-06T04:32:33.679 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = True 2023-10-06T04:32:33.679 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2023-10-06T04:32:33.679 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = False 2023-10-06T04:32:33.679 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T04:32:33.679 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.679 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:32:33.683 DEBUG:teuthology.orchestra.run.smithi177:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-10-06T04:32:33.726 INFO:teuthology.orchestra.run.smithi123.stdout:[global] 2023-10-06T04:32:33.726 INFO:teuthology.orchestra.run.smithi123.stdout: chdir = "" 2023-10-06T04:32:33.726 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:32:33.726 INFO:teuthology.orchestra.run.smithi123.stdout: auth supported = cephx 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: filestore xattr use omap = true 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: mon clock drift allowed = 1.000 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: osd crush chooseleaf type = 0 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: auth debug = true 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on old message = true 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: ms die on bug = true 2023-10-06T04:32:33.727 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: mon pg warn max object skew = 0 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default size = 2 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow primary affinity = true 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd allow pg remap = true 2023-10-06T04:32:33.728 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on no sortbitwise = false 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: mon warn on too few osds = false 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: mon_allow_pool_size_one = true 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: osd default data pool replay window = 5 2023-10-06T04:32:33.729 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: mon allow pool delete = true 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: mon cluster log file level = debug 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: debug asserts on shutdown = true 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: mon health detail to clog = false 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: mon host = "172.21.15.123,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: fsid = 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout:[osd] 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: osd journal size = 100 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.730 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub load threshold = 5.0 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd scrub max interval = 600 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd recover clone overlap = true 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd recovery max chunk = 1048576 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug shutdown = true 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug op order = true 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug verify stray on activate = true 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.731 INFO:teuthology.orchestra.run.smithi123.stdout: osd open classes on start = true 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug pg log writeout = true 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: osd map max advance = 10 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: journal zero on create = true 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: filestore apply finisher threads = 3 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.732 INFO:teuthology.orchestra.run.smithi123.stdout: bdev debug aio = true 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: osd debug misdirected ops = true 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: bdev async discard = True 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: bdev enable discard = True 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore allocator = bitmap 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore block size = 96636764160 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: bluestore fsck on mount = True 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluefs = 1/20 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: debug bluestore = 1/20 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: debug osd = 20 2023-10-06T04:32:33.733 INFO:teuthology.orchestra.run.smithi123.stdout: debug rocksdb = 4/10 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd full ratio = 0.9 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout: osd objectstore = bluestore 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.734 INFO:teuthology.orchestra.run.smithi123.stdout:[mgr] 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: debug mgr = 20 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: mgr/telemetry/nag = false 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout:[mon] 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.735 INFO:teuthology.orchestra.run.smithi123.stdout: debug mon = 20 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: debug paxos = 20 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: debug auth = 20 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: mon data avail warn = 5 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: mon mgr mkfs grace = 240 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd reporter subtree level = osd 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: mon osd prime pg temp = true 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:32:33.736 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: # 1m isn't quite enough 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: mon op complaint time = 120 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout:[client] 2023-10-06T04:32:33.737 INFO:teuthology.orchestra.run.smithi123.stdout: rgw cache enabled = true 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable ops log = true 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: rgw enable usage log = true 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: client mount timeout = 600 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: debug client = 20 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: pid file = "" 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.a] 2023-10-06T04:32:33.738 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.b] 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout:[mon.c] 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.a] 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.c] 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.b] 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout:[mds.d] 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout:[mds] 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout: debug mds = 20 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout: debug ms = 1 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug frag = True 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout: mds debug scatterstat = False 2023-10-06T04:32:33.739 INFO:teuthology.orchestra.run.smithi123.stdout: mds op complaint time = 180 2023-10-06T04:32:33.740 INFO:teuthology.orchestra.run.smithi123.stdout: mds verify scatter = False 2023-10-06T04:32:33.740 INFO:teuthology.orchestra.run.smithi123.stdout: osd op complaint time = 180 2023-10-06T04:32:33.740 INFO:teuthology.orchestra.run.smithi123.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.740 INFO:teuthology.orchestra.run.smithi123.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.748 INFO:teuthology.orchestra.run.smithi177.stdout:[global] 2023-10-06T04:32:33.748 INFO:teuthology.orchestra.run.smithi177.stdout: chdir = "" 2023-10-06T04:32:33.748 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: auth supported = cephx 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: filestore xattr use omap = true 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: mon clock drift allowed = 1.000 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: osd crush chooseleaf type = 0 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: auth debug = true 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on old message = true 2023-10-06T04:32:33.749 INFO:teuthology.orchestra.run.smithi177.stdout: ms die on bug = true 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: mon max pg per osd = 10000 # >= luminous 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: mon pg warn max object skew = 0 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: # disable pg_autoscaler by default for new pools 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: osd_pool_default_pg_autoscale_mode = off 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default size = 2 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow primary affinity = true 2023-10-06T04:32:33.750 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd allow pg remap = true 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on legacy crush tunables = false 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on crush straw calc version zero = false 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on no sortbitwise = false 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on osd down out interval zero = false 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon warn on too few osds = false 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_pool_no_redundancy = false 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: mon_allow_pool_size_one = true 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-10-06T04:32:33.751 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: osd default data pool replay window = 5 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: mon allow pool delete = true 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: mon cluster log file level = debug 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: debug asserts on shutdown = true 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: mon health detail to clog = false 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: mon host = "172.21.15.123,172.21.15.177,[v2:172.21.15.177:3301,v1:172.21.15.177:6790]" 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: fsid = 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.752 INFO:teuthology.orchestra.run.smithi177.stdout:[osd] 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: osd journal size = 100 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub load threshold = 5.0 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: osd scrub max interval = 600 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: osd mclock profile = high_recovery_ops 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: osd recover clone overlap = true 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: osd recovery max chunk = 1048576 2023-10-06T04:32:33.753 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug shutdown = true 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug op order = true 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug verify stray on activate = true 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: osd open classes on start = true 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug pg log writeout = true 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: osd deep scrub update digest min age = 30 2023-10-06T04:32:33.754 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: osd map max advance = 10 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: journal zero on create = true 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: filestore ondisk finisher threads = 3 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: filestore apply finisher threads = 3 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: bdev debug aio = true 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: osd debug misdirected ops = true 2023-10-06T04:32:33.755 INFO:teuthology.orchestra.run.smithi177.stdout: bdev async discard = True 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: bdev enable discard = True 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore allocator = bitmap 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore block size = 96636764160 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: bluestore fsck on mount = True 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluefs = 1/20 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: debug bluestore = 1/20 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: debug osd = 20 2023-10-06T04:32:33.756 INFO:teuthology.orchestra.run.smithi177.stdout: debug rocksdb = 4/10 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd backfillfull_ratio = 0.85 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd full ratio = 0.9 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd nearfull ratio = 0.8 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: osd failsafe full ratio = 0.95 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: osd objectstore = bluestore 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout:[mgr] 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: debug mgr = 20 2023-10-06T04:32:33.757 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: mgr/telemetry/nag = false 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout:[mon] 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: debug mon = 20 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: debug paxos = 20 2023-10-06T04:32:33.758 INFO:teuthology.orchestra.run.smithi177.stdout: debug auth = 20 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: mon data avail warn = 5 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: mon mgr mkfs grace = 240 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min pgs per osd = 4 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd reporter subtree level = osd 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: mon osd prime pg temp = true 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: mon reweight min bytes per osd = 10 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-10-06T04:32:33.759 INFO:teuthology.orchestra.run.smithi177.stdout: auth mon ticket ttl = 660 # 11m 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: auth service ticket ttl = 240 # 4m 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: # don't complain about insecure global_id in the test suite 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: # 1m isn't quite enough 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: mon_down_mkfs_grace = 2m 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.760 INFO:teuthology.orchestra.run.smithi177.stdout: mon_warn_on_filestore_osds = false 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: mon op complaint time = 120 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout:[client] 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: rgw cache enabled = true 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable ops log = true 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: rgw enable usage log = true 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: client mount timeout = 600 2023-10-06T04:32:33.761 INFO:teuthology.orchestra.run.smithi177.stdout: debug client = 20 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout: pid file = "" 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.a] 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.b] 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:[mon.c] 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.a] 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.c] 2023-10-06T04:32:33.762 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.b] 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout:[mds.d] 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout:[mds] 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: debug mds = 20 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: debug ms = 1 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug frag = True 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: mds debug scatterstat = False 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: mds op complaint time = 180 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: mds verify scatter = False 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: osd op complaint time = 180 2023-10-06T04:32:33.763 INFO:teuthology.orchestra.run.smithi177.stdout: rados mon op timeout = 900 2023-10-06T04:32:33.764 INFO:teuthology.orchestra.run.smithi177.stdout: rados osd op timeout = 900 2023-10-06T04:32:33.765 INFO:tasks.ceph.mds.a:Restarting daemon 2023-10-06T04:32:33.765 INFO:tasks.ceph.mds.a:Stopping old one... 2023-10-06T04:32:33.765 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:32:33.765 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:33.765 INFO:tasks.ceph.mds.c:Restarting daemon 2023-10-06T04:32:33.766 INFO:tasks.ceph.mds.c:Stopping old one... 2023-10-06T04:32:33.766 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:32:33.766 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:33.766 INFO:tasks.ceph.mds.b:Restarting daemon 2023-10-06T04:32:33.766 INFO:tasks.ceph.mds.b:Stopping old one... 2023-10-06T04:32:33.766 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:32:33.766 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:33.767 INFO:tasks.ceph.mds.d:Restarting daemon 2023-10-06T04:32:33.767 INFO:tasks.ceph.mds.d:Stopping old one... 2023-10-06T04:32:33.767 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:32:33.767 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:32:33.777 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:32:33.777 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-10-06T04:32:33.778 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:32:33.778 DEBUG:teuthology.orchestra.run.smithi177:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-10-06T04:32:33.795 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:32:33.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-10-06T04:32:33.796 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:32:33.796 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-10-06T04:32:33.819 INFO:tasks.ceph.mds.b:Started 2023-10-06T04:32:33.819 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:33.820 INFO:tasks.ceph.mds.d:Started 2023-10-06T04:32:33.820 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:33.837 INFO:tasks.ceph.mds.c:Started 2023-10-06T04:32:33.838 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:33.838 INFO:tasks.ceph.mds.a:Started 2023-10-06T04:32:33.838 DEBUG:teuthology.parallel:result is None 2023-10-06T04:32:33.839 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --cluster ceph purge cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.876+0000 7f5bd798a040 1 Processor -- start 2023-10-06T04:32:33.879 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.876+0000 7f5bd798a040 1 -- start start 2023-10-06T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd798a040 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a1778e0 0x55630a22c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd798a040 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 0x55630a1a11c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd798a040 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a248390 0x55630a253f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd798a040 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630a0bf980 con 0x55630a1778e0 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd798a040 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630a0bf620 con 0x55630a248390 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd798a040 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55630a0b6400 con 0x55630a247fc0 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a1778e0 0x55630a22c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a1778e0 0x55630a22c6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48136/0 (socket says 172.21.15.123:48136) 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/2532969640 learned_addr learned my addr 172.21.15.123:0/2532969640 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 0x55630a1a11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a248390 0x55630a253f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 msgr2=0x55630a1a11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 0x55630a1a11c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a248390 msgr2=0x55630a253f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a248390 0x55630a253f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/2532969640 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bbc005db0 con 0x55630a1778e0 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.877+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a248390 0x55630a253f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a1778e0 0x55630a22c6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4001e50 tx=0x7f5bc40018c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aed0b920d714270c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bcaffd700 1 -- 172.21.15.123:0/2532969640 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4014070 con 0x55630a1778e0 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bcaffd700 1 -- 172.21.15.123:0/2532969640 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4008030 con 0x55630a1778e0 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bcaffd700 1 -- 172.21.15.123:0/2532969640 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc400be10 con 0x55630a1778e0 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 -- 172.21.15.123:0/2532969640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a1778e0 msgr2=0x55630a22c6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a1778e0 0x55630a22c6d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4001e50 tx=0x7f5bc40018c0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 -- 172.21.15.123:0/2532969640 shutdown_connections 2023-10-06T04:32:33.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a1778e0 0x55630a22c6d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a248390 0x55630a253f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 --2- 172.21.15.123:0/2532969640 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 0x55630a1a11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 -- 172.21.15.123:0/2532969640 >> 172.21.15.123:0/2532969640 conn(0x55630a1bda90 msgr2=0x55630a1a0ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 -- 172.21.15.123:0/2532969640 shutdown_connections 2023-10-06T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.878+0000 7f5bd798a040 1 -- 172.21.15.123:0/2532969640 wait complete. 2023-10-06T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.879+0000 7f5bd798a040 1 Processor -- start 2023-10-06T04:32:33.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.879+0000 7f5bd798a040 1 -- start start 2023-10-06T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a1778e0 0x55630a22a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 0x55630a1a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a248390 0x55630a242e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630a1b88d0 con 0x55630a248390 2023-10-06T04:32:33.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55630a1b8b70 con 0x55630a1778e0 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55630a1b8e10 con 0x55630a247fc0 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a248390 0x55630a242e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a248390 0x55630a242e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48148/0 (socket says 172.21.15.123:48148) 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 learned_addr learned my addr 172.21.15.123:0/1306437763 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 0x55630a1a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a1778e0 0x55630a22a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 msgr2=0x55630a1a3af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55630a247fc0 0x55630a1a3af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a1778e0 msgr2=0x55630a22a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55630a1778e0 0x55630a22a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc400a040 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a248390 0x55630a242e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00e590 tx=0x7f5bcc007040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bc97fa700 1 -- 172.21.15.123:0/1306437763 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc013460 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bc97fa700 1 -- 172.21.15.123:0/1306437763 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bcc0135c0 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bc97fa700 1 -- 172.21.15.123:0/1306437763 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bcc013980 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55630a0ce3d0 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.880+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55630a0c1ae0 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.882+0000 7f5bc97fa700 1 -- 172.21.15.123:0/1306437763 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5bcc014410 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.882+0000 7f5bc97fa700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5bb40658c0 0x7f5bb4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.882+0000 7f5bc97fa700 1 -- 172.21.15.123:0/1306437763 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f5bcc083b40 con 0x55630a248390 2023-10-06T04:32:33.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.883+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5bb40658c0 0x7f5bb4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.889 INFO:teuthology.orchestra.run.smithi123.stdout:Warning: using slow linear search 2023-10-06T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.883+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55630a24bc60 0x55630a268390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.883+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:1 5.0 5:00000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55630a242520 con 0x55630a24bc60 2023-10-06T04:32:33.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.883+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55630a24bc60 0x55630a268390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.883+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5bb40658c0 0x7f5bb4067d70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000fd0 tx=0x7f5bc4014000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.883+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55630a24bc60 0x55630a268390 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.884+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== osd_op_reply(1 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.884+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f5bbc007030 0x7f5bbc01e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.884+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:2 5.10 5:08000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55630a242520 con 0x7f5bbc007030 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.884+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f5bbc007030 0x7f5bbc01e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.884+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f5bbc007030 0x7f5bbc01e940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.885+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 1 ==== osd_op_reply(2 [pgnls start_epoch 35 out=177b] v34'10 uv10 ondisk = 1) v8 ==== 144+0+177 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.885+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f5bc401c960 0x7f5bc402b440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.885+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:3 5.8 5:10000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55630a242520 con 0x7f5bc401c960 2023-10-06T04:32:33.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.885+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f5bc401c960 0x7f5bc402b440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.885+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f5bc401c960 0x7f5bc402b440 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.886+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 1 ==== osd_op_reply(3 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.886+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f5bbc012b10 0x7f5bbc0118c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.886+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:4 5.18 5:18000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55630a242520 con 0x7f5bbc012b10 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.886+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f5bbc012b10 0x7f5bbc0118c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.886+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f5bbc012b10 0x7f5bbc0118c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.888+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.4 v2:172.21.15.177:6808/119946 1 ==== osd_op_reply(4 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc012b10 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.888+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:5 5.4 5:20000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x55630a242520 con 0x7f5bbc007030 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.889+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 2 ==== osd_op_reply(5 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.889+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:6 5.14 5:28000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bc4026040 con 0x55630a24bc60 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.889+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== osd_op_reply(6 [pgnls start_epoch 35 out=149b] v34'18 uv18 ondisk = 1) v8 ==== 144+0+149 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.889+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f5bbc01b8b0 0x7f5bbc0063a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.889+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:7 5.c 5:30000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bc4026040 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.889+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f5bbc01b8b0 0x7f5bbc0063a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.890+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f5bbc01b8b0 0x7f5bbc0063a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.895 INFO:tasks.ceph.mds.b.smithi177.stdout:starting mds.b at 2023-10-06T04:32:33.895 INFO:tasks.ceph.mds.d.smithi177.stdout:starting mds.d at 2023-10-06T04:32:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.894+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== osd_op_reply(7 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.894+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:8 5.1c 5:38000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bc4026040 con 0x7f5bbc012b10 2023-10-06T04:32:33.896 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.894+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.4 v2:172.21.15.177:6808/119946 2 ==== osd_op_reply(8 [pgnls start_epoch 35 out=73b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc012b10 2023-10-06T04:32:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.894+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7f5bcc01d030 0x7f5bcc059860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.894+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:9 5.2 5:40000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bc4026040 con 0x7f5bcc01d030 2023-10-06T04:32:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.894+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7f5bcc01d030 0x7f5bcc059860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.897 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.895+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7f5bcc01d030 0x7f5bcc059860 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.895+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== osd_op_reply(9 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bcc01d030 2023-10-06T04:32:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.895+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:10 5.12 5:48000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bc4026040 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.896+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.1 v2:172.21.15.123:6816/119720 2 ==== osd_op_reply(10 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.898 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.896+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:11 5.a 5:50000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bc4026040 con 0x7f5bc401c960 2023-10-06T04:32:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.896+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 2 ==== osd_op_reply(11 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.896+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:12 5.1a 5:58000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc007030 2023-10-06T04:32:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.897+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 3 ==== osd_op_reply(12 [pgnls start_epoch 35 out=49b] v33'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.897+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:13 5.6 5:60000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bc401c960 2023-10-06T04:32:33.899 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.897+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 3 ==== osd_op_reply(13 [pgnls start_epoch 35 out=77b] v34'2 uv2 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.897+0000 7f5bcbfff700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f5bbc031980 0x7f5bbc03d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.897+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:14 5.16 5:68000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc031980 2023-10-06T04:32:33.900 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.897+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f5bbc031980 0x7f5bbc03d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.898+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f5bbc031980 0x7f5bbc03d3f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.898+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 1 ==== osd_op_reply(14 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.898+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:15 5.e 5:70000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc012b10 2023-10-06T04:32:33.901 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.898+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.4 v2:172.21.15.177:6808/119946 3 ==== osd_op_reply(15 [pgnls start_epoch 35 out=73b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc012b10 2023-10-06T04:32:33.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.898+0000 7f5bd13a9700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f5bcc01a2b0 0x7f5bcc070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:33.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.898+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:16 5.1e 5:78000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bcc01a2b0 2023-10-06T04:32:33.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.898+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f5bcc01a2b0 0x7f5bcc070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:33.902 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.899+0000 7f5bd0ba8700 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f5bcc01a2b0 0x7f5bcc070050 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:33.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.901+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(16 [pgnls start_epoch 35 out=49b] v33'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bcc01a2b0 2023-10-06T04:32:33.903 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.901+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:17 5.1 5:80000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc012b10 2023-10-06T04:32:33.905 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.901+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.4 v2:172.21.15.177:6808/119946 4 ==== osd_op_reply(17 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc012b10 2023-10-06T04:32:33.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.901+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:18 5.11 5:88000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bcc01d030 2023-10-06T04:32:33.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.902+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.6 v2:172.21.15.177:6816/119976 2 ==== osd_op_reply(18 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bcc01d030 2023-10-06T04:32:33.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.902+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:19 5.9 5:90000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc007030 2023-10-06T04:32:33.906 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.902+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 4 ==== osd_op_reply(19 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.902+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:20 5.19 5:98000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.902+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.1 v2:172.21.15.123:6816/119720 3 ==== osd_op_reply(20 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.902+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:21 5.5 5:a0000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bcc01a2b0 2023-10-06T04:32:33.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.903+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== osd_op_reply(21 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bcc01a2b0 2023-10-06T04:32:33.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.903+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:22 5.15 5:a8000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc031980 2023-10-06T04:32:33.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.903+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 2 ==== osd_op_reply(22 [pgnls start_epoch 35 out=97b] v33'3 uv3 ondisk = 1) v8 ==== 144+0+97 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.907 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.903+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:23 5.d 5:b0000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bc401c960 2023-10-06T04:32:33.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.903+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 4 ==== osd_op_reply(23 [pgnls start_epoch 35 out=74b] v33'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.908 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.903+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:24 5.1d 5:b8000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.911 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.904+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.1 v2:172.21.15.123:6816/119720 4 ==== osd_op_reply(24 [pgnls start_epoch 35 out=73b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.904+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:25 5.3 5:c0000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bcc01a2b0 2023-10-06T04:32:33.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.904+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.0 v2:172.21.15.123:6808/119718 3 ==== osd_op_reply(25 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bcc01a2b0 2023-10-06T04:32:33.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.904+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:26 5.13 5:c8000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x55630a24bc60 2023-10-06T04:32:33.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.904+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 3 ==== osd_op_reply(26 [pgnls start_epoch 35 out=98b] v34'3 uv3 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.912 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.904+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:27 5.b 5:d0000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bc401c960 2023-10-06T04:32:33.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.905+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 5 ==== osd_op_reply(27 [pgnls start_epoch 35 out=76b] v34'3 uv3 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.905+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:28 5.1b 5:d8000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc031980 2023-10-06T04:32:33.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.905+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 3 ==== osd_op_reply(28 [pgnls start_epoch 35 out=49b] v33'1 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.905+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:29 5.7 5:e0000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc031980 2023-10-06T04:32:33.913 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.905+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 4 ==== osd_op_reply(29 [pgnls start_epoch 35 out=73b] v33'2 uv2 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.905+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:30 5.17 5:e8000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x55630a24bc60 2023-10-06T04:32:33.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 4 ==== osd_op_reply(30 [pgnls start_epoch 35 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:31 5.f 5:f0000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bbc031980 2023-10-06T04:32:33.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 5 ==== osd_op_reply(31 [pgnls start_epoch 35 out=73b] v33'1 uv1 ondisk = 1) v8 ==== 144+0+73 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:32 5.1f 5:f8000000:%01:::head [pgnls start_epoch 35 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e35) v8 -- 0x7f5bbc03d040 con 0x7f5bcc01d030 2023-10-06T04:32:33.914 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.6 v2:172.21.15.177:6816/119976 3 ==== osd_op_reply(32 [pgnls start_epoch 35 out=101b] v34'6 uv6 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bcc01d030 2023-10-06T04:32:33.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:33 5.10 5:09401a81:::601.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a2654f0 con 0x7f5bbc007030 2023-10-06T04:32:33.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:34 5.10 5:0bd6d154:::602.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a2688d0 con 0x7f5bbc007030 2023-10-06T04:32:33.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:35 5.10 5:0cf6ad0f:::10000000000.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a269170 con 0x7f5bbc007030 2023-10-06T04:32:33.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:36 5.10 5:0d82a743:::600.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a269a10 con 0x7f5bbc007030 2023-10-06T04:32:33.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.906+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:37 5.10 5:0d89b25e:::603.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26a2b0 con 0x7f5bbc007030 2023-10-06T04:32:33.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:38 5.14 5:287034c4:::1.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26ab50 con 0x55630a24bc60 2023-10-06T04:32:33.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:39 5.14 5:292cf221:::200.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26b3f0 con 0x55630a24bc60 2023-10-06T04:32:33.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:40 5.14 5:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26bc90 con 0x55630a24bc60 2023-10-06T04:32:33.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:41 5.14 5:2f55791f:::606.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26c630 con 0x55630a24bc60 2023-10-06T04:32:33.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:42 5.1c 5:3ed09add:::607.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26d190 con 0x7f5bbc012b10 2023-10-06T04:32:33.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:43 5.6 5:654134d2:::mds0_openfiles.0:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26dd80 con 0x7f5bc401c960 2023-10-06T04:32:33.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:44 5.e 5:717a0223:::608.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26e8f0 con 0x7f5bbc012b10 2023-10-06T04:32:33.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:45 5.15 5:a93a17c2:::604.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26f450 con 0x7f5bbc031980 2023-10-06T04:32:33.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:46 5.15 5:aa448500:::500.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26ff90 con 0x7f5bbc031980 2023-10-06T04:32:33.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:47 5.d 5:b50e409b:::mds_snaptable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a270b20 con 0x7f5bc401c960 2023-10-06T04:32:33.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.907+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:48 5.1d 5:b871830b:::605.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a271660 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 5 ==== osd_op_reply(33 601.00000000 [delete] v35'11 uv11 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 5 ==== osd_op_reply(38 1.00000000.inode [delete] v35'19 uv19 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 6 ==== osd_op_reply(34 602.00000000 [delete] v35'12 uv12 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 6 ==== osd_op_reply(43 mds0_openfiles.0 [delete] v35'3 uv3 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:49 5.13 5:c91d4a1d:::mds0_inotable:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a2658d0 con 0x55630a24bc60 2023-10-06T04:32:33.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:50 5.13 5:cd5a64a3:::100.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26a660 con 0x55630a24bc60 2023-10-06T04:32:33.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:51 5.b 5:d0630e4c:::mds0_sessionmap:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a271f80 con 0x7f5bc401c960 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.4 v2:172.21.15.177:6808/119946 5 ==== osd_op_reply(44 608.00000000 [delete] v35'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc012b10 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:52 5.7 5:e0b41d06:::609.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a2658d0 con 0x7f5bbc031980 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 7 ==== osd_op_reply(35 10000000000.00000000 [delete] v35'13 uv13 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:53 5.f 5:f1f69726:::400.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a26a660 con 0x7f5bbc031980 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:54 5.1f 5:fe07bba1:::100.00000000.inode:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a271f80 con 0x7f5bcc01d030 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.909+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 7 ==== osd_op_reply(47 mds_snaptable [delete] v35'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.910+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 6 ==== osd_op_reply(45 604.00000000 [delete] v35'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.910+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:55 5.1f 5:ff5b34d6:::1.00000000:head [delete] snapc 0=[] ondisk+write+known_if_redirected+full_try+supports_pool_eio e35) v8 -- 0x55630a271f80 con 0x7f5bcc01d030 2023-10-06T04:32:33.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.910+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 7 ==== osd_op_reply(46 500.00000000 [delete] v35'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.910+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.1 v2:172.21.15.123:6816/119720 5 ==== osd_op_reply(48 605.00000000 [delete] v35'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc01b8b0 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.910+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 8 ==== osd_op_reply(36 600.00000000 [delete] v35'14 uv14 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.910+0000 7f5bd0ba8700 1 -- 172.21.15.123:0/1306437763 <== osd.7 v2:172.21.15.177:6824/119977 9 ==== osd_op_reply(37 603.00000000 [delete] v35'15 uv15 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bc4016070 con 0x7f5bbc007030 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.911+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.6 v2:172.21.15.177:6816/119976 4 ==== osd_op_reply(54 100.00000000.inode [delete] v35'7 uv7 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bcc01d030 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.911+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 6 ==== osd_op_reply(39 200.00000000 [delete] v35'20 uv20 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.911+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.4 v2:172.21.15.177:6808/119946 6 ==== osd_op_reply(42 607.00000000 [delete] v35'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc012b10 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.911+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 7 ==== osd_op_reply(40 200.00000001 [delete] v35'21 uv21 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.911+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 8 ==== osd_op_reply(41 606.00000000 [delete] v35'22 uv22 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.912+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.2 v2:172.21.15.123:6824/119723 8 ==== osd_op_reply(51 mds0_sessionmap [delete] v35'4 uv4 ondisk = 0) v8 ==== 159+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bc401c960 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.912+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 8 ==== osd_op_reply(52 609.00000000 [delete] v35'3 uv3 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.912+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 <== osd.5 v2:172.21.15.177:6800/119947 9 ==== osd_op_reply(53 400.00000000 [delete] v35'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bcc052c60 con 0x7f5bbc031980 2023-10-06T04:32:33.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.914+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 9 ==== osd_op_reply(49 mds0_inotable [delete] v35'4 uv4 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.915+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.6 v2:172.21.15.177:6816/119976 5 ==== osd_op_reply(55 1.00000000 [delete] v35'8 uv8 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x7f5bcc01d030 2023-10-06T04:32:33.923 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2023-10-06T04:32:33.923 INFO:tasks.ceph.mds.c.smithi123.stdout:starting mds.c at 2023-10-06T04:32:33.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bcbfff700 1 -- 172.21.15.123:0/1306437763 <== osd.3 v2:172.21.15.123:6800/119719 10 ==== osd_op_reply(50 100.00000000 [delete] v35'5 uv5 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x7f5bbc008ab0 con 0x55630a24bc60 2023-10-06T04:32:33.924 INFO:teuthology.orchestra.run.smithi123.stdout:Removed 23 objects 2023-10-06T04:32:33.924 INFO:teuthology.orchestra.run.smithi123.stdout:successfully purged pool cephfs_metadata 2023-10-06T04:32:33.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f5bcc01a2b0 msgr2=0x7f5bcc070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f5bcc01a2b0 0x7f5bcc070050 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f5bbc01b8b0 msgr2=0x7f5bbc0063a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f5bbc01b8b0 0x7f5bbc0063a0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f5bc401c960 msgr2=0x7f5bc402b440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f5bc401c960 0x7f5bc402b440 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55630a24bc60 msgr2=0x55630a268390 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 reap_dead start 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55630a24bc60 0x55630a268390 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f5bbc012b10 msgr2=0x7f5bbc0118c0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f5bbc012b10 0x7f5bbc0118c0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f5bbc031980 msgr2=0x7f5bbc03d3f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7f5bbc031980 0x7f5bbc03d3f0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7f5bcc01d030 msgr2=0x7f5bcc059860 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7f5bcc01d030 0x7f5bcc059860 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f5bbc007030 msgr2=0x7f5bbc01e940 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f5bbc007030 0x7f5bbc01e940 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd13a9700 1 -- 172.21.15.123:0/1306437763 reap_dead start 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5bb40658c0 msgr2=0x7f5bb4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5bb40658c0 0x7f5bb4067d70 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f5bc4000fd0 tx=0x7f5bc4014000 comp rx=0 tx=0).stop 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a248390 msgr2=0x55630a242e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a248390 0x55630a242e10 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f5bcc00e590 tx=0x7f5bcc007040 comp rx=0 tx=0).stop 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 shutdown_connections 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55630a248390 0x55630a242e10 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 --2- 172.21.15.123:0/1306437763 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5bb40658c0 0x7f5bb4067d70 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:33.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 >> 172.21.15.123:0/1306437763 conn(0x55630a1bda90 msgr2=0x55630a22c660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:33.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 shutdown_connections 2023-10-06T04:32:33.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:33.917+0000 7f5bd798a040 1 -- 172.21.15.123:0/1306437763 wait complete. 2023-10-06T04:32:33.928 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:34.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.081+0000 7f21e15e0700 1 Processor -- start 2023-10-06T04:32:34.083 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.081+0000 7f21e15e0700 1 -- start start 2023-10-06T04:32:34.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.081+0000 7f21e15e0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f21dc0ac720 0x7f21dc0ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.081+0000 7f21e15e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc06abb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.081+0000 7f21e15e0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21dc0aeea0 con 0x7f21dc0680b0 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.081+0000 7f21e15e0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21dc06deb0 con 0x7f21dc0ac720 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21e15e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0aecc0 con 0x7f21dc06a750 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21db7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc06abb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21daffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f21dc0ac720 0x7f21dc0ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21db7fe700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc06abb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:47240/0 (socket says 172.21.15.123:47240) 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21db7fe700 1 -- 172.21.15.123:0/417547134 learned_addr learned my addr 172.21.15.123:0/417547134 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21daffd700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f21dc0ac720 0x7f21dc0ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:43608/0 (socket says 172.21.15.123:43608) 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21daffd700 1 -- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 msgr2=0x7f21dc06abb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:34.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21daffd700 1 --2- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc06abb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21daffd700 1 -- 172.21.15.123:0/417547134 >> v1:172.21.15.177:6789/0 conn(0x7f21dc0680b0 legacy=0x7f21dc0acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21daffd700 1 -- 172.21.15.123:0/417547134 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21dc0af180 con 0x7f21dc0ac720 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21db7fe700 1 --2- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21daffd700 1 --2- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 0x7f21dc0ae780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0098e0 tx=0x7f21cc0091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=de6542739a4a0625 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21d9ffb700 1 -- 172.21.15.123:0/417547134 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00b6d0 con 0x7f21dc0ac720 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21d9ffb700 1 -- 172.21.15.123:0/417547134 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc00b830 con 0x7f21dc0ac720 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.082+0000 7f21d9ffb700 1 -- 172.21.15.123:0/417547134 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00bbd0 con 0x7f21dc0ac720 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 -- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 msgr2=0x7f21dc0ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 --2- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 0x7f21dc0ae780 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0098e0 tx=0x7f21cc0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 -- 172.21.15.123:0/417547134 shutdown_connections 2023-10-06T04:32:34.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 --2- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc06abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 --2- 172.21.15.123:0/417547134 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 0x7f21dc0ae780 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 -- 172.21.15.123:0/417547134 >> 172.21.15.123:0/417547134 conn(0x7f21dc0cda80 msgr2=0x7f21dc0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:34.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 -- 172.21.15.123:0/417547134 shutdown_connections 2023-10-06T04:32:34.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.083+0000 7f21e15e0700 1 -- 172.21.15.123:0/417547134 wait complete. 2023-10-06T04:32:34.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.084+0000 7f21e15e0700 1 Processor -- start 2023-10-06T04:32:34.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.084+0000 7f21e15e0700 1 -- start start 2023-10-06T04:32:34.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21e15e0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21dc0680b0 0x7f21dc168080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21e15e0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc1614a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21e15e0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 0x7f21dc0b1dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21e15e0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc1685c0 con 0x7f21dc0680b0 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21e15e0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0c1b60 con 0x7f21dc0ac720 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21e15e0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f21dc0c0f50 con 0x7f21dc06a750 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21dc0680b0 0x7f21dc168080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21dc0680b0 0x7f21dc168080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48202/0 (socket says 172.21.15.123:48202) 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 -- 172.21.15.123:0/873408770 learned_addr learned my addr 172.21.15.123:0/873408770 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:34.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21db7fe700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 0x7f21dc0b1dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21da7fc700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc1614a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 -- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 msgr2=0x7f21dc1614a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc1614a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 -- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 msgr2=0x7f21dc0b1dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 0x7f21dc0b1dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.085+0000 7f21daffd700 1 -- 172.21.15.123:0/873408770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21d4009ae0 con 0x7f21dc0680b0 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.086+0000 7f21daffd700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21dc0680b0 0x7f21dc168080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0c0a20 tx=0x7f21cc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.086+0000 7f21c2ffd700 1 -- 172.21.15.123:0/873408770 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc01bcc0 con 0x7f21dc0680b0 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.086+0000 7f21c2ffd700 1 -- 172.21.15.123:0/873408770 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21cc01be60 con 0x7f21dc0680b0 2023-10-06T04:32:34.092 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.086+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f21cc009420 con 0x7f21dc0680b0 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.086+0000 7f21c2ffd700 1 -- 172.21.15.123:0/873408770 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21cc011670 con 0x7f21dc0680b0 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.086+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f21dc0c1510 con 0x7f21dc0680b0 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.087+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21dc0cde80 con 0x7f21dc0680b0 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.087+0000 7f21c2ffd700 1 -- 172.21.15.123:0/873408770 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f21cc0119e0 con 0x7f21dc0680b0 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.087+0000 7f21c2ffd700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f21c8066f30 0x7f21c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.088+0000 7f21c2ffd700 1 -- 172.21.15.123:0/873408770 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f21cc085f10 con 0x7f21dc0680b0 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.088+0000 7f21da7fc700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f21c8066f30 0x7f21c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.093 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.088+0000 7f21da7fc700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f21c8066f30 0x7f21c80693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f21dc153890 tx=0x7f21d0007ec0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.094 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.092+0000 7f21c2ffd700 1 -- 172.21.15.123:0/873408770 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f21cc052fd0 con 0x7f21dc0680b0 2023-10-06T04:32:34.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.280+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f21dc0af460 con 0x7f21dc0680b0 2023-10-06T04:32:34.283 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.280+0000 7f21c2ffd700 1 -- 172.21.15.123:0/873408770 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v39) v1 ==== 74+0+17141 (secure 0 0 0) 0x7f21cc0565f0 con 0x7f21dc0680b0 2023-10-06T04:32:34.283 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:34.283 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":39,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:34.263343+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":6,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":32,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:34.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.282+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f21c8066f30 msgr2=0x7f21c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.282+0000 7f21e15e0700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f21c8066f30 0x7f21c80693e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f21dc153890 tx=0x7f21d0007ec0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.282+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21dc0680b0 msgr2=0x7f21dc168080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.282+0000 7f21e15e0700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21dc0680b0 0x7f21dc168080 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f21dc0c0a20 tx=0x7f21cc017000 comp rx=0 tx=0).stop 2023-10-06T04:32:34.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 shutdown_connections 2023-10-06T04:32:34.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f21c8066f30 0x7f21c80693e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f21dc0680b0 0x7f21dc168080 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f21dc0ac720 0x7f21dc0b1dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 --2- 172.21.15.123:0/873408770 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f21dc06a750 0x7f21dc1614a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 >> 172.21.15.123:0/873408770 conn(0x7f21dc0cda80 msgr2=0x7f21dc0c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:34.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 shutdown_connections 2023-10-06T04:32:34.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.283+0000 7f21e15e0700 1 -- 172.21.15.123:0/873408770 wait complete. 2023-10-06T04:32:34.297 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:32:21.900854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:32:22.782310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2023-10-06T04:32:34.298 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.416+0000 7f748fca5700 1 Processor -- start 2023-10-06T04:32:34.418 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.416+0000 7f748fca5700 1 -- start start 2023-10-06T04:32:34.420 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748fca5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8340 0x7f74880a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748fca5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f74880c8710 0x7f74880c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748fca5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7488069aa0 con 0x7f7488068670 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748fca5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f748806a340 con 0x7f74880c8710 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748fca5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74880aa6f0 con 0x7f74880c8340 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748da41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8340 0x7f74880a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748d240700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f74880c8710 0x7f74880c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748e242700 1 --1- >> v1:172.21.15.177:6789/0 conn(0x7f7488068670 0x7f74880c6d40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.177:6789/0 says I am v1:172.21.15.123:56922/0 (socket says 172.21.15.123:56922) 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748d240700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f74880c8710 0x7f74880c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:43630/0 (socket says 172.21.15.123:43630) 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748da41700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8340 0x7f74880a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:47266/0 (socket says 172.21.15.123:47266) 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748e242700 1 -- 172.21.15.123:0/4134978420 learned_addr learned my addr 172.21.15.123:0/4134978420 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:34.421 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748da41700 1 -- 172.21.15.123:0/4134978420 >> v2:172.21.15.177:3300/0 conn(0x7f74880c8710 msgr2=0x7f74880c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748da41700 1 --2- 172.21.15.123:0/4134978420 >> v2:172.21.15.177:3300/0 conn(0x7f74880c8710 0x7f74880c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748da41700 1 -- 172.21.15.123:0/4134978420 >> v1:172.21.15.177:6789/0 conn(0x7f7488068670 legacy=0x7f74880c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748da41700 1 -- 172.21.15.123:0/4134978420 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7488069cf0 con 0x7f74880c8340 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748ca3f700 1 -- 172.21.15.123:0/4134978420 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3682550888 0 0) 0x7f7488069aa0 con 0x7f7488068670 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748d240700 1 --2- 172.21.15.123:0/4134978420 >> v2:172.21.15.177:3300/0 conn(0x7f74880c8710 0x7f74880c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.417+0000 7f748da41700 1 --2- 172.21.15.123:0/4134978420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8340 0x7f74880a9df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7484009910 tx=0x7f74840091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4fb680f7eec1880d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748ca3f700 1 -- 172.21.15.123:0/4134978420 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400b640 con 0x7f74880c8340 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748ca3f700 1 -- 172.21.15.123:0/4134978420 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f748400b7a0 con 0x7f74880c8340 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748ca3f700 1 -- 172.21.15.123:0/4134978420 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400bb60 con 0x7f74880c8340 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 -- 172.21.15.123:0/4134978420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8340 msgr2=0x7f74880a9df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 --2- 172.21.15.123:0/4134978420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8340 0x7f74880a9df0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f7484009910 tx=0x7f74840091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 -- 172.21.15.123:0/4134978420 shutdown_connections 2023-10-06T04:32:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 --2- 172.21.15.123:0/4134978420 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8340 0x7f74880a9df0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 --2- 172.21.15.123:0/4134978420 >> v2:172.21.15.177:3300/0 conn(0x7f74880c8710 0x7f74880c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 -- 172.21.15.123:0/4134978420 >> 172.21.15.123:0/4134978420 conn(0x7f74880c9ca0 msgr2=0x7f748806da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 -- 172.21.15.123:0/4134978420 shutdown_connections 2023-10-06T04:32:34.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.418+0000 7f748fca5700 1 -- 172.21.15.123:0/4134978420 wait complete. 2023-10-06T04:32:34.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.419+0000 7f748fca5700 1 Processor -- start 2023-10-06T04:32:34.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.419+0000 7f748fca5700 1 -- start start 2023-10-06T04:32:34.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748fca5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7488068670 0x7f748806aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748fca5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74880c8340 0x7f7488156aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748fca5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8710 0x7f7488156fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748fca5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c49d0 con 0x7f7488068670 2023-10-06T04:32:34.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748fca5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74880c4c70 con 0x7f74880c8340 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748fca5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74881699a0 con 0x7f74880c8710 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748da41700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7488068670 0x7f748806aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748da41700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7488068670 0x7f748806aa20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48210/0 (socket says 172.21.15.123:48210) 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748da41700 1 -- 172.21.15.123:0/529023073 learned_addr learned my addr 172.21.15.123:0/529023073 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748d240700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74880c8340 0x7f7488156aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748e242700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8710 0x7f7488156fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748da41700 1 -- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8710 msgr2=0x7f7488156fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748da41700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8710 0x7f7488156fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748da41700 1 -- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74880c8340 msgr2=0x7f7488156aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.420+0000 7f748da41700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74880c8340 0x7f7488156aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f748da41700 1 -- 172.21.15.123:0/529023073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7478009420 con 0x7f7488068670 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f748d240700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74880c8340 0x7f7488156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f748da41700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7488068670 0x7f748806aa20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7484009fb0 tx=0x7f7484017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f747dffb700 1 -- 172.21.15.123:0/529023073 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748401f070 con 0x7f7488068670 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f747dffb700 1 -- 172.21.15.123:0/529023073 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7484000bc0 con 0x7f7488068670 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7484009420 con 0x7f7488068670 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f747dffb700 1 -- 172.21.15.123:0/529023073 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f748400bb60 con 0x7f7488068670 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.421+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74880b12b0 con 0x7f7488068670 2023-10-06T04:32:34.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.422+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7488069fd0 con 0x7f7488068670 2023-10-06T04:32:34.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.423+0000 7f747dffb700 1 -- 172.21.15.123:0/529023073 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f7484023070 con 0x7f7488068670 2023-10-06T04:32:34.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.423+0000 7f747dffb700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f746c066f30 0x7f746c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.424+0000 7f747dffb700 1 -- 172.21.15.123:0/529023073 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f7484085f40 con 0x7f7488068670 2023-10-06T04:32:34.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.424+0000 7f748d240700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f746c066f30 0x7f746c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.424+0000 7f748d240700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f746c066f30 0x7f746c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74880c36e0 tx=0x7f74780091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.427+0000 7f747dffb700 1 -- 172.21.15.123:0/529023073 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f7484053000 con 0x7f7488068670 2023-10-06T04:32:34.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.612+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f74880c7cc0 con 0x7f7488068670 2023-10-06T04:32:34.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.612+0000 7f747dffb700 1 -- 172.21.15.123:0/529023073 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v39) v1 ==== 93+0+10 (secure 0 0 0) 0x7f7484056620 con 0x7f7488068670 2023-10-06T04:32:34.615 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:34.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f746c066f30 msgr2=0x7f746c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f746c066f30 0x7f746c0693e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f74880c36e0 tx=0x7f74780091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7488068670 msgr2=0x7f748806aa20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7488068670 0x7f748806aa20 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f7484009fb0 tx=0x7f7484017000 comp rx=0 tx=0).stop 2023-10-06T04:32:34.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 shutdown_connections 2023-10-06T04:32:34.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f746c066f30 0x7f746c0693e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f7488068670 0x7f748806aa20 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f74880c8710 0x7f7488156fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 --2- 172.21.15.123:0/529023073 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f74880c8340 0x7f7488156aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 >> 172.21.15.123:0/529023073 conn(0x7f74880c9ca0 msgr2=0x7f7488158910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:34.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 shutdown_connections 2023-10-06T04:32:34.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.614+0000 7f748fca5700 1 -- 172.21.15.123:0/529023073 wait complete. 2023-10-06T04:32:34.628 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:34.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.745+0000 7f5831116700 1 Processor -- start 2023-10-06T04:32:34.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.745+0000 7f5831116700 1 -- start start 2023-10-06T04:32:34.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f5831116700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f582c0c8a10 0x7f582c06e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f5831116700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c06e820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f5831116700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f582c06a4e0 con 0x7f582c13fa70 2023-10-06T04:32:34.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f5831116700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f582c0c8ec0 con 0x7f582c0c8a10 2023-10-06T04:32:34.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f5831116700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0c91a0 con 0x7f582c0cabf0 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582a59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f582c0c8a10 0x7f582c06e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582a59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f582c0c8a10 0x7f582c06e2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48224/0 (socket says 172.21.15.123:48224) 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582a59c700 1 -- 172.21.15.123:0/177996107 learned_addr learned my addr 172.21.15.123:0/177996107 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582b59e700 1 --2- 172.21.15.123:0/177996107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c06e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582a59c700 1 -- 172.21.15.123:0/177996107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 msgr2=0x7f582c06e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582a59c700 1 --2- 172.21.15.123:0/177996107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c06e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582a59c700 1 -- 172.21.15.123:0/177996107 >> v1:172.21.15.177:6789/0 conn(0x7f582c13fa70 legacy=0x7f582c0c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582a59c700 1 -- 172.21.15.123:0/177996107 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f582c0c9480 con 0x7f582c0c8a10 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.746+0000 7f582b59e700 1 --2- 172.21.15.123:0/177996107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f582a59c700 1 --2- 172.21.15.123:0/177996107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c06e2e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f581c0099b0 tx=0x7f581c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eed03c9d1739a37b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5829d9b700 1 -- 172.21.15.123:0/177996107 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00b820 con 0x7f582c0c8a10 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5829d9b700 1 -- 172.21.15.123:0/177996107 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f581c00b980 con 0x7f582c0c8a10 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5829d9b700 1 -- 172.21.15.123:0/177996107 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f581c00bd20 con 0x7f582c0c8a10 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 -- 172.21.15.123:0/177996107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 msgr2=0x7f582c06e2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 --2- 172.21.15.123:0/177996107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c06e2e0 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f581c0099b0 tx=0x7f581c009250 comp rx=0 tx=0).stop 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 -- 172.21.15.123:0/177996107 shutdown_connections 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 --2- 172.21.15.123:0/177996107 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c06e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 --2- 172.21.15.123:0/177996107 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c06e2e0 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 -- 172.21.15.123:0/177996107 >> 172.21.15.123:0/177996107 conn(0x7f582c0cd020 msgr2=0x7f582c0cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 -- 172.21.15.123:0/177996107 shutdown_connections 2023-10-06T04:32:34.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.747+0000 7f5831116700 1 -- 172.21.15.123:0/177996107 wait complete. 2023-10-06T04:32:34.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 Processor -- start 2023-10-06T04:32:34.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 -- start start 2023-10-06T04:32:34.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c0c4a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c0ad800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f582c13fa70 0x7f582c13c9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0afde0 con 0x7f582c0c8a10 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0b0030 con 0x7f582c13fa70 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f5831116700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f582c0b0310 con 0x7f582c0cabf0 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f582ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c0c4a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f582ad9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c0c4a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48228/0 (socket says 172.21.15.123:48228) 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f582ad9d700 1 -- 172.21.15.123:0/2674829784 learned_addr learned my addr 172.21.15.123:0/2674829784 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.749+0000 7f582b59e700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f582c13fa70 0x7f582c13c9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.756 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f582a59c700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c0ad800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f582ad9d700 1 -- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 msgr2=0x7f582c0ad800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f582ad9d700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c0ad800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f582ad9d700 1 -- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f582c13fa70 msgr2=0x7f582c13c9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f582ad9d700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f582c13fa70 0x7f582c13c9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f582ad9d700 1 -- 172.21.15.123:0/2674829784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f581c0094f0 con 0x7f582c0c8a10 2023-10-06T04:32:34.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f582ad9d700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c0c4a20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f582000c5d0 tx=0x7f5820007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f5812ffd700 1 -- 172.21.15.123:0/2674829784 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820011590 con 0x7f582c0c8a10 2023-10-06T04:32:34.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f5812ffd700 1 -- 172.21.15.123:0/2674829784 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5820011710 con 0x7f582c0c8a10 2023-10-06T04:32:34.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f582c0b05f0 con 0x7f582c0c8a10 2023-10-06T04:32:34.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f5812ffd700 1 -- 172.21.15.123:0/2674829784 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5820011ad0 con 0x7f582c0c8a10 2023-10-06T04:32:34.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.750+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f582c0c4f60 con 0x7f582c0c8a10 2023-10-06T04:32:34.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.751+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f582c0c5150 con 0x7f582c0c8a10 2023-10-06T04:32:34.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.751+0000 7f5812ffd700 1 -- 172.21.15.123:0/2674829784 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f582001c460 con 0x7f582c0c8a10 2023-10-06T04:32:34.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.752+0000 7f5812ffd700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:34.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.752+0000 7f5812ffd700 1 -- 172.21.15.123:0/2674829784 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f58200843a0 con 0x7f582c0c8a10 2023-10-06T04:32:34.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.752+0000 7f582a59c700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:34.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.752+0000 7f582a59c700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5818066f30 0x7f58180693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f581c0099b0 tx=0x7f581c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:34.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.756+0000 7f5812ffd700 1 -- 172.21.15.123:0/2674829784 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5820051460 con 0x7f582c0c8a10 2023-10-06T04:32:34.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.935+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f582c0c1dc0 con 0x7f582c0c8a10 2023-10-06T04:32:34.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.936+0000 7f5812ffd700 1 -- 172.21.15.123:0/2674829784 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v39) v1 ==== 92+0+10 (secure 0 0 0) 0x7f5820054a80 con 0x7f582c0c8a10 2023-10-06T04:32:34.938 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:34.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5818066f30 msgr2=0x7f58180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5818066f30 0x7f58180693e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f581c0099b0 tx=0x7f581c016000 comp rx=0 tx=0).stop 2023-10-06T04:32:34.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 msgr2=0x7f582c0c4a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c0c4a20 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f582000c5d0 tx=0x7f5820007f60 comp rx=0 tx=0).stop 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 shutdown_connections 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5818066f30 0x7f58180693e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f582c0c8a10 0x7f582c0c4a20 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f582c13fa70 0x7f582c13c9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 --2- 172.21.15.123:0/2674829784 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f582c0cabf0 0x7f582c0ad800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 >> 172.21.15.123:0/2674829784 conn(0x7f582c0cd020 msgr2=0x7f582c13dc30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 shutdown_connections 2023-10-06T04:32:34.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:34.938+0000 7f5831116700 1 -- 172.21.15.123:0/2674829784 wait complete. 2023-10-06T04:32:34.952 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:32:35.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.069+0000 7f9a5219d700 1 Processor -- start 2023-10-06T04:32:35.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.069+0000 7f9a5219d700 1 -- start start 2023-10-06T04:32:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a5219d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9a4c1597d0 0x7f9a4c068770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a5219d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c0cab40 0x7f9a4c0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a5219d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a4c0c9640 con 0x7f9a4c0cc220 2023-10-06T04:32:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a5219d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a4c0cd530 con 0x7f9a4c1597d0 2023-10-06T04:32:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a5219d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c0ccd90 con 0x7f9a4c0cab40 2023-10-06T04:32:35.076 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a4b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9a4c1597d0 0x7f9a4c068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a4bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9a4c0cc220 0x7f9a4c0cc680 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33216/0 (socket says 172.21.15.123:33216) 2023-10-06T04:32:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a4b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9a4c1597d0 0x7f9a4c068770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48230/0 (socket says 172.21.15.123:48230) 2023-10-06T04:32:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.070+0000 7f9a4bfff700 1 -- 172.21.15.123:0/2596606269 learned_addr learned my addr 172.21.15.123:0/2596606269 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4affd700 1 --2- 172.21.15.123:0/2596606269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c0cab40 0x7f9a4c0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4b7fe700 1 -- 172.21.15.123:0/2596606269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c0cab40 msgr2=0x7f9a4c0caf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4b7fe700 1 --2- 172.21.15.123:0/2596606269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c0cab40 0x7f9a4c0caf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4b7fe700 1 -- 172.21.15.123:0/2596606269 >> v1:172.21.15.123:6789/0 conn(0x7f9a4c0cc220 legacy=0x7f9a4c0cc680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4b7fe700 1 -- 172.21.15.123:0/2596606269 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a4c0cb4d0 con 0x7f9a4c1597d0 2023-10-06T04:32:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4a7fc700 1 -- 172.21.15.123:0/2596606269 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2161420849 0 0) 0x7f9a4c0c9640 con 0x7f9a4c0cc220 2023-10-06T04:32:35.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4affd700 1 --2- 172.21.15.123:0/2596606269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c0cab40 0x7f9a4c0caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4b7fe700 1 --2- 172.21.15.123:0/2596606269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c1597d0 0x7f9a4c068770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9a400098d0 tx=0x7f9a40009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea410626fcf57eb2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4a7fc700 1 -- 172.21.15.123:0/2596606269 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a40009e70 con 0x7f9a4c1597d0 2023-10-06T04:32:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4a7fc700 1 -- 172.21.15.123:0/2596606269 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a40005c10 con 0x7f9a4c1597d0 2023-10-06T04:32:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.071+0000 7f9a4a7fc700 1 -- 172.21.15.123:0/2596606269 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a4000b650 con 0x7f9a4c1597d0 2023-10-06T04:32:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 -- 172.21.15.123:0/2596606269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c1597d0 msgr2=0x7f9a4c068770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 --2- 172.21.15.123:0/2596606269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c1597d0 0x7f9a4c068770 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f9a400098d0 tx=0x7f9a40009120 comp rx=0 tx=0).stop 2023-10-06T04:32:35.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 -- 172.21.15.123:0/2596606269 shutdown_connections 2023-10-06T04:32:35.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 --2- 172.21.15.123:0/2596606269 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c1597d0 0x7f9a4c068770 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 --2- 172.21.15.123:0/2596606269 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c0cab40 0x7f9a4c0caf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 -- 172.21.15.123:0/2596606269 >> 172.21.15.123:0/2596606269 conn(0x7f9a4c097450 msgr2=0x7f9a4c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:35.080 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 -- 172.21.15.123:0/2596606269 shutdown_connections 2023-10-06T04:32:35.081 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.072+0000 7f9a5219d700 1 -- 172.21.15.123:0/2596606269 wait complete. 2023-10-06T04:32:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.073+0000 7f9a5219d700 1 Processor -- start 2023-10-06T04:32:35.084 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.073+0000 7f9a5219d700 1 -- start start 2023-10-06T04:32:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a5219d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4c0cab40 0x7f9a4c152720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a5219d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c0cc220 0x7f9a4c1561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a5219d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c1597d0 0x7f9a4c1566f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a5219d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c16a960 con 0x7f9a4c0cc220 2023-10-06T04:32:35.085 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a5219d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c152c60 con 0x7f9a4c0cab40 2023-10-06T04:32:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a5219d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a4c169d90 con 0x7f9a4c1597d0 2023-10-06T04:32:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a4affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c0cc220 0x7f9a4c1561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a4affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c0cc220 0x7f9a4c1561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48244/0 (socket says 172.21.15.123:48244) 2023-10-06T04:32:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.074+0000 7f9a4affd700 1 -- 172.21.15.123:0/546701163 learned_addr learned my addr 172.21.15.123:0/546701163 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:35.086 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4bfff700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c1597d0 0x7f9a4c1566f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4b7fe700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4c0cab40 0x7f9a4c152720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4affd700 1 -- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c1597d0 msgr2=0x7f9a4c1566f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4affd700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c1597d0 0x7f9a4c1566f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.087 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4affd700 1 -- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4c0cab40 msgr2=0x7f9a4c152720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4affd700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4c0cab40 0x7f9a4c152720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4affd700 1 -- 172.21.15.123:0/546701163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a400093c0 con 0x7f9a4c0cc220 2023-10-06T04:32:35.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4b7fe700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4c0cab40 0x7f9a4c152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:35.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a4affd700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c0cc220 0x7f9a4c1561b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00c850 tx=0x7f9a3c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a337fe700 1 -- 172.21.15.123:0/546701163 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c011600 con 0x7f9a4c0cc220 2023-10-06T04:32:35.088 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a337fe700 1 -- 172.21.15.123:0/546701163 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a3c011760 con 0x7f9a4c0cc220 2023-10-06T04:32:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a4c16a030 con 0x7f9a4c0cc220 2023-10-06T04:32:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a337fe700 1 -- 172.21.15.123:0/546701163 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a3c011b20 con 0x7f9a4c0cc220 2023-10-06T04:32:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.075+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a4c16a5f0 con 0x7f9a4c0cc220 2023-10-06T04:32:35.089 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.076+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a4c0c8a20 con 0x7f9a4c0cc220 2023-10-06T04:32:35.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.077+0000 7f9a337fe700 1 -- 172.21.15.123:0/546701163 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9a3c018350 con 0x7f9a4c0cc220 2023-10-06T04:32:35.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.077+0000 7f9a337fe700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9a38066f30 0x7f9a380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.077+0000 7f9a337fe700 1 -- 172.21.15.123:0/546701163 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f9a3c084600 con 0x7f9a4c0cc220 2023-10-06T04:32:35.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.077+0000 7f9a4b7fe700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9a38066f30 0x7f9a380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.090 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.077+0000 7f9a4b7fe700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9a38066f30 0x7f9a380693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009160 tx=0x7f9a40016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.091 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.081+0000 7f9a337fe700 1 -- 172.21.15.123:0/546701163 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9a3c055150 con 0x7f9a4c0cc220 2023-10-06T04:32:35.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.262+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9a4c06d840 con 0x7f9a4c0cc220 2023-10-06T04:32:35.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.262+0000 7f9a337fe700 1 -- 172.21.15.123:0/546701163 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v39) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9a3c054ce0 con 0x7f9a4c0cc220 2023-10-06T04:32:35.265 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:35.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9a38066f30 msgr2=0x7f9a380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9a38066f30 0x7f9a380693e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f9a40009160 tx=0x7f9a40016000 comp rx=0 tx=0).stop 2023-10-06T04:32:35.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c0cc220 msgr2=0x7f9a4c1561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c0cc220 0x7f9a4c1561b0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f9a3c00c850 tx=0x7f9a3c008040 comp rx=0 tx=0).stop 2023-10-06T04:32:35.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 shutdown_connections 2023-10-06T04:32:35.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9a38066f30 0x7f9a380693e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9a4c1597d0 0x7f9a4c1566f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9a4c0cab40 0x7f9a4c152720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 --2- 172.21.15.123:0/546701163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9a4c0cc220 0x7f9a4c1561b0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 >> 172.21.15.123:0/546701163 conn(0x7f9a4c097450 msgr2=0x7f9a4c069e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:35.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 shutdown_connections 2023-10-06T04:32:35.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.264+0000 7f9a5219d700 1 -- 172.21.15.123:0/546701163 wait complete. 2023-10-06T04:32:35.279 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:32:35.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.398+0000 7fa087952700 1 Processor -- start 2023-10-06T04:32:35.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.399+0000 7fa087952700 1 -- start start 2023-10-06T04:32:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.399+0000 7fa087952700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 0x7fa0800c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa087952700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa080069aa0 con 0x7fa0800c8710 2023-10-06T04:32:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa087952700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa08006a340 con 0x7fa0800c8340 2023-10-06T04:32:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa087952700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0800aa500 con 0x7fa080068d80 2023-10-06T04:32:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa084eed700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa0800c8710 0x7fa080068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33222/0 (socket says 172.21.15.123:33222) 2023-10-06T04:32:35.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa084eed700 1 -- 172.21.15.123:0/580913176 learned_addr learned my addr 172.21.15.123:0/580913176 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2144050490 0 0) 0x7fa080069aa0 con 0x7fa0800c8710 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa06c003570 con 0x7fa0800c8710 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa085eef700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 0x7fa0800c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1609937260 0 0) 0x7fa06c003570 con 0x7fa0800c8710 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 msgr2=0x7fa0800c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 0x7fa0800c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 >> v1:172.21.15.177:6789/0 conn(0x7fa0800c8340 legacy=0x7fa0800a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa0800aa6f0 con 0x7fa0800c8710 2023-10-06T04:32:35.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fa07c001470 con 0x7fa0800c8710 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa085eef700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 0x7fa0800c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 >> v1:172.21.15.123:6789/0 conn(0x7fa0800c8710 legacy=0x7fa080068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06c003a90 0x7fa06c003ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c004810 0x7fa06c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 0x7fa06c005250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c00bbf0 con 0x7fa06c003a90 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c00bd30 con 0x7fa06c004810 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa06c00bef0 con 0x7fa080068d80 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa0856ee700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06c003a90 0x7fa06c003ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa084eed700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c004810 0x7fa06c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.400+0000 7fa085eef700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 0x7fa06c005250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa0856ee700 1 -- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 msgr2=0x7fa06c005250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa0856ee700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa080068d80 0x7fa06c005250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa0856ee700 1 -- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c004810 msgr2=0x7fa06c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa0856ee700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c004810 0x7fa06c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa0856ee700 1 -- 172.21.15.123:0/580913176 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa06c003570 con 0x7fa06c003a90 2023-10-06T04:32:35.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa084eed700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c004810 0x7fa06c004c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa0856ee700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06c003a90 0x7fa06c003ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0700018b0 tx=0x7fa070007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa3ef0588b3295e8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070012420 con 0x7fa06c003a90 2023-10-06T04:32:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa070012580 con 0x7fa06c003a90 2023-10-06T04:32:35.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa077fff700 1 -- 172.21.15.123:0/580913176 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa070012920 con 0x7fa06c003a90 2023-10-06T04:32:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa087952700 1 -- 172.21.15.123:0/580913176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06c003a90 msgr2=0x7fa06c003ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa087952700 1 --2- 172.21.15.123:0/580913176 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa06c003a90 0x7fa06c003ea0 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7fa0700018b0 tx=0x7fa070007d30 comp rx=0 tx=0).stop 2023-10-06T04:32:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa085eef700 1 -- 172.21.15.123:0/580913176 reap_dead start 2023-10-06T04:32:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa087952700 1 -- 172.21.15.123:0/580913176 shutdown_connections 2023-10-06T04:32:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.401+0000 7fa087952700 1 -- 172.21.15.123:0/580913176 >> 172.21.15.123:0/580913176 conn(0x7fa0800c9ca0 msgr2=0x7fa08006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.402+0000 7fa087952700 1 -- 172.21.15.123:0/580913176 shutdown_connections 2023-10-06T04:32:35.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.402+0000 7fa087952700 1 -- 172.21.15.123:0/580913176 wait complete. 2023-10-06T04:32:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.403+0000 7fa087952700 1 Processor -- start 2023-10-06T04:32:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.403+0000 7fa087952700 1 -- start start 2023-10-06T04:32:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa087952700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c003a90 0x7fa08006bce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa087952700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa08006a6d0 0x7fa08015e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.413 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa087952700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0800af630 0x7fa0800afab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa087952700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa080150140 con 0x7fa0800af630 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa087952700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa08006ac40 con 0x7fa06c003a90 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa087952700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa08006af20 con 0x7fa08006a6d0 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0800af630 0x7fa0800afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0800af630 0x7fa0800afab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48268/0 (socket says 172.21.15.123:48268) 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 -- 172.21.15.123:0/200595669 learned_addr learned my addr 172.21.15.123:0/200595669 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa0856ee700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c003a90 0x7fa08006bce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 -- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa08006a6d0 msgr2=0x7fa08015e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa084eed700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa08006a6d0 0x7fa08015e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa08006a6d0 0x7fa08015e9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.414 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 -- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c003a90 msgr2=0x7fa08006bce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c003a90 0x7fa08006bce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa085eef700 1 -- 172.21.15.123:0/200595669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa08006b200 con 0x7fa0800af630 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.404+0000 7fa0856ee700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c003a90 0x7fa08006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.405+0000 7fa085eef700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0800af630 0x7fa0800afab0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa0800c6cf0 tx=0x7fa078009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.405+0000 7fa0757fa700 1 -- 172.21.15.123:0/200595669 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078002ce0 con 0x7fa0800af630 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.405+0000 7fa0757fa700 1 -- 172.21.15.123:0/200595669 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07800f050 con 0x7fa0800af630 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.405+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa08006b4e0 con 0x7fa0800af630 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.405+0000 7fa0757fa700 1 -- 172.21.15.123:0/200595669 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa078014660 con 0x7fa0800af630 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.405+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa08015eef0 con 0x7fa0800af630 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.406+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa0800ca0a0 con 0x7fa0800af630 2023-10-06T04:32:35.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.406+0000 7fa0757fa700 1 -- 172.21.15.123:0/200595669 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa078003040 con 0x7fa0800af630 2023-10-06T04:32:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.406+0000 7fa0757fa700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa064066f30 0x7fa0640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.406+0000 7fa0757fa700 1 -- 172.21.15.123:0/200595669 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6144+0+0 (secure 0 0 0) 0x7fa0780848f0 con 0x7fa0800af630 2023-10-06T04:32:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.407+0000 7fa0856ee700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa064066f30 0x7fa0640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.407+0000 7fa0856ee700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa064066f30 0x7fa0640693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa080152740 tx=0x7fa070015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.416 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.411+0000 7fa0757fa700 1 -- 172.21.15.123:0/200595669 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa0780519b0 con 0x7fa0800af630 2023-10-06T04:32:35.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.592+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa080069cf0 con 0x7fa0800af630 2023-10-06T04:32:35.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.593+0000 7fa0757fa700 1 -- 172.21.15.123:0/200595669 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v39) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa078054fd0 con 0x7fa0800af630 2023-10-06T04:32:35.595 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:32:35.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa064066f30 msgr2=0x7fa0640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa064066f30 0x7fa0640693e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fa080152740 tx=0x7fa070015000 comp rx=0 tx=0).stop 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0800af630 msgr2=0x7fa0800afab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0800af630 0x7fa0800afab0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7fa0800c6cf0 tx=0x7fa078009870 comp rx=0 tx=0).stop 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 shutdown_connections 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa064066f30 0x7fa0640693e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa0800af630 0x7fa0800afab0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa06c003a90 0x7fa08006bce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 --2- 172.21.15.123:0/200595669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa08006a6d0 0x7fa08015e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 >> 172.21.15.123:0/200595669 conn(0x7fa0800c9ca0 msgr2=0x7fa08015f210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:35.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 shutdown_connections 2023-10-06T04:32:35.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.595+0000 7fa087952700 1 -- 172.21.15.123:0/200595669 wait complete. 2023-10-06T04:32:35.609 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'recovery_fs' 2023-10-06T04:32:35.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create recovery_fs_metadata --pg_num_min 64 2023-10-06T04:32:35.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.729+0000 7f1defd43700 1 Processor -- start 2023-10-06T04:32:35.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.730+0000 7f1defd43700 1 -- start start 2023-10-06T04:32:35.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.730+0000 7f1defd43700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1de8155f90 0x7f1de806de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.730+0000 7f1defd43700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de806e720 con 0x7f1de80c6ee0 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.730+0000 7f1defd43700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1de806ea00 con 0x7f1de8156360 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.730+0000 7f1defd43700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1de806ece0 con 0x7f1de8155f90 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1dee2e0700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1de80c6ee0 0x7f1de80c9e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33228/0 (socket says 172.21.15.123:33228) 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1dee2e0700 1 -- 172.21.15.123:0/1876451594 learned_addr learned my addr 172.21.15.123:0/1876451594 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1876451594 >> v2:172.21.15.177:3300/0 conn(0x7f1de8155f90 0x7f1de806de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 531705752 0 0) 0x7f1de806e720 con 0x7f1de80c6ee0 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1dd4003570 con 0x7f1de80c6ee0 2023-10-06T04:32:35.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2170119640 0 0) 0x7f1dd4003570 con 0x7f1de80c6ee0 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 >> v2:172.21.15.177:3300/0 conn(0x7f1de8155f90 msgr2=0x7f1de806de40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 --2- 172.21.15.123:0/1876451594 >> v2:172.21.15.177:3300/0 conn(0x7f1de8155f90 0x7f1de806de40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 >> v1:172.21.15.177:6789/0 conn(0x7f1de8156360 legacy=0x7f1de80c67d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de806efc0 con 0x7f1de80c6ee0 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f1de00017f0 con 0x7f1de80c6ee0 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 >> v1:172.21.15.123:6789/0 conn(0x7f1de80c6ee0 legacy=0x7f1de80c9e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1dd4003e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd4004810 0x7f1dd4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd40051c0 0x7f1dd400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de806ece0 con 0x7f1dd4003a60 2023-10-06T04:32:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd400c050 con 0x7f1dd40051c0 2023-10-06T04:32:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1dd400e420 con 0x7f1dd4004810 2023-10-06T04:32:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1dd4003e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1dee2e0700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd40051c0 0x7f1dd400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.731+0000 7f1ded2de700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd4004810 0x7f1dd4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1dedadf700 1 -- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd4004810 msgr2=0x7f1dd4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1dd4004810 0x7f1dd4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1dedadf700 1 -- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd40051c0 msgr2=0x7f1dd400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1dd40051c0 0x7f1dd400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1dee2e0700 1 -- 172.21.15.123:0/1876451594 reap_dead start 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1dedadf700 1 -- 172.21.15.123:0/1876451594 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1dd4003570 con 0x7f1dd4003a60 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1dd4003e30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009be0 tx=0x7f1dd80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7374210bcfd9d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800e710 con 0x7f1dd4003a60 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd800e870 con 0x7f1dd4003a60 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1decadd700 1 -- 172.21.15.123:0/1876451594 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800ec30 con 0x7f1dd4003a60 2023-10-06T04:32:35.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1defd43700 1 -- 172.21.15.123:0/1876451594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 msgr2=0x7f1dd4003e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1defd43700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1dd4003e30 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8009be0 tx=0x7f1dd80091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1defd43700 1 -- 172.21.15.123:0/1876451594 shutdown_connections 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1defd43700 1 --2- 172.21.15.123:0/1876451594 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1dd4003e30 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1defd43700 1 -- 172.21.15.123:0/1876451594 >> 172.21.15.123:0/1876451594 conn(0x7f1de80cbee0 msgr2=0x7f1de80cc2e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1defd43700 1 -- 172.21.15.123:0/1876451594 shutdown_connections 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.732+0000 7f1defd43700 1 -- 172.21.15.123:0/1876451594 wait complete. 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.733+0000 7f1defd43700 1 Processor -- start 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1defd43700 1 -- start start 2023-10-06T04:32:35.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1defd43700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1de80c6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1defd43700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80c31c0 0x7f1de814e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1defd43700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80cabb0 0x7f1de80cb000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1defd43700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de814ff80 con 0x7f1dd4003a60 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1defd43700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80cb540 con 0x7f1de80c31c0 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1defd43700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1de80c3c80 con 0x7f1de80cabb0 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1dedadf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1de80c6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1dedadf700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1de80c6030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48272/0 (socket says 172.21.15.123:48272) 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1dedadf700 1 -- 172.21.15.123:0/1622377571 learned_addr learned my addr 172.21.15.123:0/1622377571 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1ded2de700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80c31c0 0x7f1de814e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.734+0000 7f1dee2e0700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80cabb0 0x7f1de80cb000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dedadf700 1 -- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80cabb0 msgr2=0x7f1de80cb000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80cabb0 0x7f1de80cb000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dedadf700 1 -- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80c31c0 msgr2=0x7f1de814e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80c31c0 0x7f1de814e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dedadf700 1 -- 172.21.15.123:0/1622377571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1de80c39a0 con 0x7f1dd4003a60 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dedadf700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1de80c6030 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8013a70 tx=0x7f1dd8019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dddffb700 1 -- 172.21.15.123:0/1622377571 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd800e710 con 0x7f1dd4003a60 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dddffb700 1 -- 172.21.15.123:0/1622377571 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1dd8021040 con 0x7f1dd4003a60 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1de814f930 con 0x7f1dd4003a60 2023-10-06T04:32:35.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1dddffb700 1 -- 172.21.15.123:0/1622377571 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1dd8013ea0 con 0x7f1dd4003a60 2023-10-06T04:32:35.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.735+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1de80c3630 con 0x7f1dd4003a60 2023-10-06T04:32:35.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.736+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1de8150260 con 0x7f1dd4003a60 2023-10-06T04:32:35.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.736+0000 7f1dddffb700 1 -- 172.21.15.123:0/1622377571 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1dd8025850 con 0x7f1dd4003a60 2023-10-06T04:32:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.737+0000 7f1dddffb700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc066f30 0x7f1dcc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.737+0000 7f1dddffb700 1 -- 172.21.15.123:0/1622377571 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6144+0+0 (secure 0 0 0) 0x7f1dd8085d40 con 0x7f1dd4003a60 2023-10-06T04:32:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.737+0000 7f1ded2de700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc066f30 0x7f1dcc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.737+0000 7f1ded2de700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc066f30 0x7f1dcc0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1de80ca150 tx=0x7f1de4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:35.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.741+0000 7f1dddffb700 1 -- 172.21.15.123:0/1622377571 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1dd8052e00 con 0x7f1dd4003a60 2023-10-06T04:32:35.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:35.922+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f1de81521a0 con 0x7f1dd4003a60 2023-10-06T04:32:36.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.587+0000 7f1dddffb700 1 -- 172.21.15.123:0/1622377571 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "recovery_fs_metadata", "pg_num_min": 64}]=0 pool 'recovery_fs_metadata' created v40) v1 ==== 148+0+0 (secure 0 0 0) 0x7f1dd8056420 con 0x7f1dd4003a60 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc066f30 msgr2=0x7f1dcc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc066f30 0x7f1dcc0693e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f1de80ca150 tx=0x7f1de4009040 comp rx=0 tx=0).stop 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 msgr2=0x7f1de80c6030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1de80c6030 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7f1dd8013a70 tx=0x7f1dd8019000 comp rx=0 tx=0).stop 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 shutdown_connections 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1dcc066f30 0x7f1dcc0693e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1dd4003a60 0x7f1de80c6030 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1de80cabb0 0x7f1de80cb000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 --2- 172.21.15.123:0/1622377571 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1de80c31c0 0x7f1de814e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 >> 172.21.15.123:0/1622377571 conn(0x7f1de80cbee0 msgr2=0x7f1de80c2570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:36.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.592+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 shutdown_connections 2023-10-06T04:32:36.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.593+0000 7f1defd43700 1 -- 172.21.15.123:0/1622377571 wait complete. 2023-10-06T04:32:36.597 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'recovery_fs_metadata' created 2023-10-06T04:32:36.610 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-10-06T04:32:36.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.734+0000 7effa5fc7700 1 Processor -- start 2023-10-06T04:32:36.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7effa5fc7700 1 -- start start 2023-10-06T04:32:36.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7effa5fc7700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effa0157430 0x7effa00ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:36.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7effa5fc7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa0159690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:36.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7effa5fc7700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effa00c9d70 con 0x7effa00caf00 2023-10-06T04:32:36.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7effa5fc7700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effa0157800 con 0x7effa0157430 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7effa5fc7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa0159200 con 0x7effa00c9150 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7eff9f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effa0157430 0x7effa00ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7eff9f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7effa0157430 0x7effa00ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48278/0 (socket says 172.21.15.123:48278) 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.735+0000 7eff9f7fe700 1 -- 172.21.15.123:0/1638128240 learned_addr learned my addr 172.21.15.123:0/1638128240 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9ffff700 1 --2- 172.21.15.123:0/1638128240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa0159690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9f7fe700 1 -- 172.21.15.123:0/1638128240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 msgr2=0x7effa0159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9f7fe700 1 --2- 172.21.15.123:0/1638128240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa0159690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9f7fe700 1 -- 172.21.15.123:0/1638128240 >> v1:172.21.15.177:6789/0 conn(0x7effa00caf00 legacy=0x7effa00c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9f7fe700 1 -- 172.21.15.123:0/1638128240 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effa00c9550 con 0x7effa0157430 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9f7fe700 1 --2- 172.21.15.123:0/1638128240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa0157430 0x7effa00ca9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7eff94009990 tx=0x7eff94009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef49b81664868197 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:36.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9e7fc700 1 -- 172.21.15.123:0/1638128240 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff94005c10 con 0x7effa0157430 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9e7fc700 1 -- 172.21.15.123:0/1638128240 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff94005d70 con 0x7effa0157430 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.736+0000 7eff9e7fc700 1 -- 172.21.15.123:0/1638128240 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff9400b840 con 0x7effa0157430 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 -- 172.21.15.123:0/1638128240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa0157430 msgr2=0x7effa00ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 --2- 172.21.15.123:0/1638128240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa0157430 0x7effa00ca9c0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7eff94009990 tx=0x7eff94009220 comp rx=0 tx=0).stop 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 -- 172.21.15.123:0/1638128240 shutdown_connections 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 --2- 172.21.15.123:0/1638128240 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa0159690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 --2- 172.21.15.123:0/1638128240 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa0157430 0x7effa00ca9c0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 -- 172.21.15.123:0/1638128240 >> 172.21.15.123:0/1638128240 conn(0x7effa0097450 msgr2=0x7effa0067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 -- 172.21.15.123:0/1638128240 shutdown_connections 2023-10-06T04:32:36.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.737+0000 7effa5fc7700 1 -- 172.21.15.123:0/1638128240 wait complete. 2023-10-06T04:32:36.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.738+0000 7effa5fc7700 1 Processor -- start 2023-10-06T04:32:36.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.738+0000 7effa5fc7700 1 -- start start 2023-10-06T04:32:36.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa01526b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:36.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa00caf00 0x7effa00cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:36.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0157430 0x7effa00ce000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:36.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c5480 con 0x7effa00caf00 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c1e20 con 0x7effa0157430 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effa00c2100 con 0x7effa00c9150 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa00caf00 0x7effa00cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa00caf00 0x7effa00cbcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48284/0 (socket says 172.21.15.123:48284) 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 -- 172.21.15.123:0/2155017466 learned_addr learned my addr 172.21.15.123:0/2155017466 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9ffff700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0157430 0x7effa00ce000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9f7fe700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa01526b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 -- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 msgr2=0x7effa01526b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa01526b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 -- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0157430 msgr2=0x7effa00ce000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0157430 0x7effa00ce000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 -- 172.21.15.123:0/2155017466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff94009480 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff9effd700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa00caf00 0x7effa00cbcb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c600 tx=0x7eff90007f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff877fe700 1 -- 172.21.15.123:0/2155017466 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90011590 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff877fe700 1 -- 172.21.15.123:0/2155017466 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff90011710 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7eff877fe700 1 -- 172.21.15.123:0/2155017466 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff90011ad0 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effa00c7910 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.739+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effa00c7ed0 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.740+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effa006d840 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.741+0000 7eff877fe700 1 -- 172.21.15.123:0/2155017466 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff9001c460 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.741+0000 7eff877fe700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff8c066f30 0x7eff8c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.741+0000 7eff9f7fe700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff8c066f30 0x7eff8c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.741+0000 7eff877fe700 1 -- 172.21.15.123:0/2155017466 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6519+0+0 (secure 0 0 0) 0x7eff90084640 con 0x7effa00caf00 2023-10-06T04:32:36.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.742+0000 7eff9f7fe700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff8c066f30 0x7eff8c0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7eff94003080 tx=0x7eff94015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:36.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.746+0000 7eff877fe700 1 -- 172.21.15.123:0/2155017466 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff90051580 con 0x7effa00caf00 2023-10-06T04:32:36.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.926+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7effa00c2240 con 0x7effa00caf00 2023-10-06T04:32:36.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.927+0000 7eff877fe700 1 -- 172.21.15.123:0/2155017466 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' already exists v40) v1 ==== 177+0+0 (secure 0 0 0) 0x7eff90054ba0 con 0x7effa00caf00 2023-10-06T04:32:36.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff8c066f30 msgr2=0x7eff8c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:36.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff8c066f30 0x7eff8c0693e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7eff94003080 tx=0x7eff94015000 comp rx=0 tx=0).stop 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa00caf00 msgr2=0x7effa00cbcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa00caf00 0x7effa00cbcb0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7eff9000c600 tx=0x7eff90007f60 comp rx=0 tx=0).stop 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 shutdown_connections 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff8c066f30 0x7eff8c0693e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7effa0157430 0x7effa00ce000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7effa00c9150 0x7effa01526b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 --2- 172.21.15.123:0/2155017466 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7effa00caf00 0x7effa00cbcb0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 >> 172.21.15.123:0/2155017466 conn(0x7effa0097450 msgr2=0x7effa0154df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 shutdown_connections 2023-10-06T04:32:36.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:36.929+0000 7effa5fc7700 1 -- 172.21.15.123:0/2155017466 wait complete. 2023-10-06T04:32:36.933 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' already exists 2023-10-06T04:32:36.943 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new recovery_fs recovery_fs_metadata cephfs_data --recover --allow-dangerous-metadata-overlay 2023-10-06T04:32:37.064 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.062+0000 7f9c7037a700 1 Processor -- start 2023-10-06T04:32:37.065 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.062+0000 7f9c7037a700 1 -- start start 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c7037a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9c680678b0 0x7f9c680aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c7037a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c6806add0 con 0x7f9c68159540 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c7037a700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c68068570 con 0x7f9c68067c80 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c7037a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c680af930 con 0x7f9c680678b0 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6e116700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9c680678b0 0x7f9c680aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6e917700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9c68159540 0x7f9c6806b260 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33244/0 (socket says 172.21.15.123:33244) 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6e116700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f9c680678b0 0x7f9c680aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48288/0 (socket says 172.21.15.123:48288) 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6e917700 1 -- 172.21.15.123:0/2326918476 learned_addr learned my addr 172.21.15.123:0/2326918476 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6e116700 1 -- 172.21.15.123:0/2326918476 >> v1:172.21.15.177:6789/0 conn(0x7f9c68067c80 legacy=0x7f9c68158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:37.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6e116700 1 -- 172.21.15.123:0/2326918476 >> v1:172.21.15.123:6789/0 conn(0x7f9c68159540 legacy=0x7f9c6806b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6e116700 1 -- 172.21.15.123:0/2326918476 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c6806a4a0 con 0x7f9c680678b0 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.063+0000 7f9c6d114700 1 -- 172.21.15.123:0/2326918476 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2362173366 0 0) 0x7f9c6806add0 con 0x7f9c68159540 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c6e116700 1 --2- 172.21.15.123:0/2326918476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c680678b0 0x7f9c680aec90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c580098e0 tx=0x7f9c580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13cf88836cd9167c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c6d114700 1 -- 172.21.15.123:0/2326918476 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c58009e80 con 0x7f9c680678b0 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c6d114700 1 -- 172.21.15.123:0/2326918476 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c5800b820 con 0x7f9c680678b0 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c6d114700 1 -- 172.21.15.123:0/2326918476 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c5800bba0 con 0x7f9c680678b0 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c7037a700 1 -- 172.21.15.123:0/2326918476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c680678b0 msgr2=0x7f9c680aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c7037a700 1 --2- 172.21.15.123:0/2326918476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c680678b0 0x7f9c680aec90 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f9c580098e0 tx=0x7f9c580091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c7037a700 1 -- 172.21.15.123:0/2326918476 shutdown_connections 2023-10-06T04:32:37.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c7037a700 1 --2- 172.21.15.123:0/2326918476 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c680678b0 0x7f9c680aec90 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c7037a700 1 -- 172.21.15.123:0/2326918476 >> 172.21.15.123:0/2326918476 conn(0x7f9c680cdb10 msgr2=0x7f9c680abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:37.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c7037a700 1 -- 172.21.15.123:0/2326918476 shutdown_connections 2023-10-06T04:32:37.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.064+0000 7f9c7037a700 1 -- 172.21.15.123:0/2326918476 wait complete. 2023-10-06T04:32:37.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.065+0000 7f9c7037a700 1 Processor -- start 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c7037a700 1 -- start start 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c7037a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c680678b0 0x7f9c6815bdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c7037a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c68067c80 0x7f9c68154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c7037a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c68159540 0x7f9c680b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c7037a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6815cfc0 con 0x7f9c68159540 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c7037a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6815c3b0 con 0x7f9c680678b0 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c7037a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c6815c690 con 0x7f9c68067c80 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c6e917700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c68159540 0x7f9c680b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c6e917700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c68159540 0x7f9c680b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48298/0 (socket says 172.21.15.123:48298) 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c6e917700 1 -- 172.21.15.123:0/3248583833 learned_addr learned my addr 172.21.15.123:0/3248583833 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c6e116700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c680678b0 0x7f9c6815bdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.066+0000 7f9c6d915700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c68067c80 0x7f9c68154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c6e917700 1 -- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c68067c80 msgr2=0x7f9c68154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c6e917700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c68067c80 0x7f9c68154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c6e917700 1 -- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c680678b0 msgr2=0x7f9c6815bdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c6e917700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c680678b0 0x7f9c6815bdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c6e917700 1 -- 172.21.15.123:0/3248583833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c58009420 con 0x7f9c68159540 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c6e917700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c68159540 0x7f9c680b08c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000cbf0 tx=0x7f9c600019b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c5e7fc700 1 -- 172.21.15.123:0/3248583833 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000a8b0 con 0x7f9c68159540 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c5e7fc700 1 -- 172.21.15.123:0/3248583833 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c6000aa10 con 0x7f9c68159540 2023-10-06T04:32:37.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c6815c970 con 0x7f9c68159540 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c5e7fc700 1 -- 172.21.15.123:0/3248583833 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c6000add0 con 0x7f9c68159540 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.067+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c680b0e00 con 0x7f9c68159540 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.068+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c6815d130 con 0x7f9c68159540 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.068+0000 7f9c5e7fc700 1 -- 172.21.15.123:0/3248583833 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9c60007300 con 0x7f9c68159540 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.069+0000 7f9c5e7fc700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9c4c066f30 0x7f9c4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.069+0000 7f9c5e7fc700 1 -- 172.21.15.123:0/3248583833 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(40..40 src has 1..40) v4 ==== 6519+0+0 (secure 0 0 0) 0x7f9c60084b60 con 0x7f9c68159540 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.069+0000 7f9c6e116700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9c4c066f30 0x7f9c4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.069+0000 7f9c6e116700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9c4c066f30 0x7f9c4c0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c580030c0 tx=0x7f9c58016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:37.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.073+0000 7f9c5e7fc700 1 -- 172.21.15.123:0/3248583833 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9c60051a80 con 0x7f9c68159540 2023-10-06T04:32:37.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.295+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true} v 0) v1 -- 0x7f9c6806a780 con 0x7f9c68159540 2023-10-06T04:32:37.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.603+0000 7f9c5e7fc700 1 -- 172.21.15.123:0/3248583833 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "recovery_fs", "metadata": "recovery_fs_metadata", "data": "cephfs_data", "recover": true, "allow_dangerous_metadata_overlay": true}]=0 new fs with metadata pool 7 and data pool 6 v20) v1 ==== 241+0+0 (secure 0 0 0) 0x7f9c600550a0 con 0x7f9c68159540 2023-10-06T04:32:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.605+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9c4c066f30 msgr2=0x7f9c4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.605+0000 7f9c7037a700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9c4c066f30 0x7f9c4c0693e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f9c580030c0 tx=0x7f9c58016000 comp rx=0 tx=0).stop 2023-10-06T04:32:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.605+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c68159540 msgr2=0x7f9c680b08c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.605+0000 7f9c7037a700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c68159540 0x7f9c680b08c0 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f9c6000cbf0 tx=0x7f9c600019b0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 shutdown_connections 2023-10-06T04:32:37.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9c4c066f30 0x7f9c4c0693e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9c68159540 0x7f9c680b08c0 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9c680678b0 0x7f9c6815bdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 --2- 172.21.15.123:0/3248583833 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9c68067c80 0x7f9c68154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 >> 172.21.15.123:0/3248583833 conn(0x7f9c680cdb10 msgr2=0x7f9c68157590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:37.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 shutdown_connections 2023-10-06T04:32:37.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.606+0000 7f9c7037a700 1 -- 172.21.15.123:0/3248583833 wait complete. 2023-10-06T04:32:37.609 INFO:teuthology.orchestra.run.smithi123.stderr:new fs with metadata pool 7 and data pool 6 2023-10-06T04:32:37.620 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:37.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.734+0000 7eff8bde6700 1 Processor -- start 2023-10-06T04:32:37.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.735+0000 7eff8bde6700 1 -- start start 2023-10-06T04:32:37.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.735+0000 7eff8bde6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 0x7eff840c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.735+0000 7eff8bde6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff840c8580 con 0x7eff8406e960 2023-10-06T04:32:37.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff8bde6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff840c8860 con 0x7eff8406f080 2023-10-06T04:32:37.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff8bde6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff840c8b40 con 0x7eff841560c0 2023-10-06T04:32:37.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff8a383700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7eff8406e960 0x7eff840ada60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:33248/0 (socket says 172.21.15.123:33248) 2023-10-06T04:32:37.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff8a383700 1 -- 172.21.15.123:0/4261505745 learned_addr learned my addr 172.21.15.123:0/4261505745 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff89b82700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 0x7eff840c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159693169 0 0) 0x7eff840c8580 con 0x7eff8406e960 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff70003570 con 0x7eff8406e960 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1381321394 0 0) 0x7eff70003570 con 0x7eff8406e960 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 msgr2=0x7eff840c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 0x7eff840c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 >> v1:172.21.15.177:6789/0 conn(0x7eff8406f080 legacy=0x7eff8406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff840c8e20 con 0x7eff8406e960 2023-10-06T04:32:37.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7eff7c0017f0 con 0x7eff8406e960 2023-10-06T04:32:37.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 >> v1:172.21.15.123:6789/0 conn(0x7eff8406e960 legacy=0x7eff840ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 0x7eff70003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff70003fd0 0x7eff70004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70004d50 0x7eff700051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff840c8b40 con 0x7eff70003fd0 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff7000bb40 con 0x7eff70004d50 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff7000bd00 con 0x7eff841560c0 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff89381700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff70003fd0 0x7eff70004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.736+0000 7eff89b82700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 0x7eff70003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff8a383700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70004d50 0x7eff700051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff89381700 1 -- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 msgr2=0x7eff70003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff89381700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 0x7eff70003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.745 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff89381700 1 -- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70004d50 msgr2=0x7eff700051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff89381700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70004d50 0x7eff700051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff89381700 1 -- 172.21.15.123:0/4261505745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff70003570 con 0x7eff70003fd0 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff89b82700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff841560c0 0x7eff70003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff89381700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff70003fd0 0x7eff70004810 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff800019a0 tx=0x7eff80007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6db25714781e6231 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000c5d0 con 0x7eff70003fd0 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000c750 con 0x7eff70003fd0 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff88b80700 1 -- 172.21.15.123:0/4261505745 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000cb10 con 0x7eff70003fd0 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff8bde6700 1 -- 172.21.15.123:0/4261505745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff70003fd0 msgr2=0x7eff70004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff8bde6700 1 --2- 172.21.15.123:0/4261505745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff70003fd0 0x7eff70004810 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7eff800019a0 tx=0x7eff80007d10 comp rx=0 tx=0).stop 2023-10-06T04:32:37.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff8a383700 1 -- 172.21.15.123:0/4261505745 reap_dead start 2023-10-06T04:32:37.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff8bde6700 1 -- 172.21.15.123:0/4261505745 shutdown_connections 2023-10-06T04:32:37.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.737+0000 7eff8bde6700 1 -- 172.21.15.123:0/4261505745 >> 172.21.15.123:0/4261505745 conn(0x7eff840cda80 msgr2=0x7eff840c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:37.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.738+0000 7eff8bde6700 1 -- 172.21.15.123:0/4261505745 shutdown_connections 2023-10-06T04:32:37.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.738+0000 7eff8bde6700 1 -- 172.21.15.123:0/4261505745 wait complete. 2023-10-06T04:32:37.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.739+0000 7eff8bde6700 1 Processor -- start 2023-10-06T04:32:37.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.739+0000 7eff8bde6700 1 -- start start 2023-10-06T04:32:37.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8bde6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70003fd0 0x7eff840b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8bde6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff840b0d70 0x7eff840c56c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8bde6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff840c5c00 0x7eff840c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8bde6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff840c65b0 con 0x7eff840b0d70 2023-10-06T04:32:37.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8bde6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff840b1330 con 0x7eff70003fd0 2023-10-06T04:32:37.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8bde6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff840b1610 con 0x7eff840c5c00 2023-10-06T04:32:37.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff840b0d70 0x7eff840c56c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff840b0d70 0x7eff840c56c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48314/0 (socket says 172.21.15.123:48314) 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 -- 172.21.15.123:0/2475254669 learned_addr learned my addr 172.21.15.123:0/2475254669 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89b82700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70003fd0 0x7eff840b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8a383700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff840c5c00 0x7eff840c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 -- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff840c5c00 msgr2=0x7eff840c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff840c5c00 0x7eff840c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 -- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70003fd0 msgr2=0x7eff840b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70003fd0 0x7eff840b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 -- 172.21.15.123:0/2475254669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff840b18f0 con 0x7eff840b0d70 2023-10-06T04:32:37.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89b82700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70003fd0 0x7eff840b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff8a383700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff840c5c00 0x7eff840c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.740+0000 7eff89381700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff840b0d70 0x7eff840c56c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff7000bb10 tx=0x7eff80015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.741+0000 7eff75ffb700 1 -- 172.21.15.123:0/2475254669 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000c5d0 con 0x7eff840b0d70 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.741+0000 7eff75ffb700 1 -- 172.21.15.123:0/2475254669 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8000ce80 con 0x7eff840b0d70 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.741+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff840b1bd0 con 0x7eff840b0d70 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.741+0000 7eff75ffb700 1 -- 172.21.15.123:0/2475254669 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8000cb10 con 0x7eff840b0d70 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.741+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff841539e0 con 0x7eff840b0d70 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.742+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff840b2220 con 0x7eff840b0d70 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.742+0000 7eff75ffb700 1 -- 172.21.15.123:0/2475254669 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7eff80019030 con 0x7eff840b0d70 2023-10-06T04:32:37.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.742+0000 7eff75ffb700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff68066f30 0x7eff680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:37.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.743+0000 7eff75ffb700 1 -- 172.21.15.123:0/2475254669 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6597+0+0 (secure 0 0 0) 0x7eff80085ea0 con 0x7eff840b0d70 2023-10-06T04:32:37.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.743+0000 7eff89b82700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff68066f30 0x7eff680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:37.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.743+0000 7eff89b82700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff68066f30 0x7eff680693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff78009de0 tx=0x7eff78008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:37.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.747+0000 7eff75ffb700 1 -- 172.21.15.123:0/2475254669 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7eff80052db0 con 0x7eff840b0d70 2023-10-06T04:32:37.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.928+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff840cde80 con 0x7eff840b0d70 2023-10-06T04:32:37.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.929+0000 7eff75ffb700 1 -- 172.21.15.123:0/2475254669 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18783 (secure 0 0 0) 0x7eff80017090 con 0x7eff840b0d70 2023-10-06T04:32:37.932 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:37.932 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":41,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:37.297115+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff68066f30 msgr2=0x7eff680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff68066f30 0x7eff680693e0 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7eff78009de0 tx=0x7eff78008040 comp rx=0 tx=0).stop 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff840b0d70 msgr2=0x7eff840c56c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff840b0d70 0x7eff840c56c0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7eff7000bb10 tx=0x7eff80015000 comp rx=0 tx=0).stop 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 shutdown_connections 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7eff68066f30 0x7eff680693e0 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7eff840c5c00 0x7eff840c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7eff70003fd0 0x7eff840b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 --2- 172.21.15.123:0/2475254669 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7eff840b0d70 0x7eff840c56c0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 >> 172.21.15.123:0/2475254669 conn(0x7eff840cda80 msgr2=0x7eff840ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 shutdown_connections 2023-10-06T04:32:37.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:37.931+0000 7eff8bde6700 1 -- 172.21.15.123:0/2475254669 wait complete. 2023-10-06T04:32:37.946 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:38.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.061+0000 7f3ddf8b8700 1 Processor -- start 2023-10-06T04:32:38.063 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.061+0000 7f3ddf8b8700 1 -- start start 2023-10-06T04:32:38.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddf8b8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3dd8155780 0x7f3dd8068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddf8b8700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3dd8068bb0 0x7f3dd80c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddf8b8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dd8069020 con 0x7f3dd80c7460 2023-10-06T04:32:38.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddf8b8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd80c7c50 con 0x7f3dd8155780 2023-10-06T04:32:38.066 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddf8b8700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dd80c7f30 con 0x7f3dd8068bb0 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3dd8155780 0x7f3dd8068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3dd8155780 0x7f3dd8068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48324/0 (socket says 172.21.15.123:48324) 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 -- 172.21.15.123:0/1847519820 learned_addr learned my addr 172.21.15.123:0/1847519820 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddce53700 1 --2- 172.21.15.123:0/1847519820 >> v2:172.21.15.177:3300/0 conn(0x7f3dd8068bb0 0x7f3dd80c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 -- 172.21.15.123:0/1847519820 >> v2:172.21.15.177:3300/0 conn(0x7f3dd8068bb0 msgr2=0x7f3dd80c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 --2- 172.21.15.123:0/1847519820 >> v2:172.21.15.177:3300/0 conn(0x7f3dd8068bb0 0x7f3dd80c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 -- 172.21.15.123:0/1847519820 >> v1:172.21.15.177:6789/0 conn(0x7f3dd80c7460 legacy=0x7f3dd8159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 -- 172.21.15.123:0/1847519820 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd80c8210 con 0x7f3dd8155780 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddce53700 1 --2- 172.21.15.123:0/1847519820 >> v2:172.21.15.177:3300/0 conn(0x7f3dd8068bb0 0x7f3dd80c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:38.067 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3ddd654700 1 --2- 172.21.15.123:0/1847519820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd8155780 0x7f3dd8068670 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3dd806ea70 tx=0x7f3dd4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b11e9da85d898f14 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3dcffff700 1 -- 172.21.15.123:0/1847519820 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400b820 con 0x7f3dd8155780 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.062+0000 7f3dcffff700 1 -- 172.21.15.123:0/1847519820 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dd400b980 con 0x7f3dd8155780 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3dcffff700 1 -- 172.21.15.123:0/1847519820 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dd400bd20 con 0x7f3dd8155780 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/1847519820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd8155780 msgr2=0x7f3dd8068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/1847519820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd8155780 0x7f3dd8068670 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f3dd806ea70 tx=0x7f3dd4009250 comp rx=0 tx=0).stop 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/1847519820 shutdown_connections 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/1847519820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd8155780 0x7f3dd8068670 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/1847519820 >> v2:172.21.15.177:3300/0 conn(0x7f3dd8068bb0 0x7f3dd80c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/1847519820 >> 172.21.15.123:0/1847519820 conn(0x7f3dd80c9c90 msgr2=0x7f3dd806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/1847519820 shutdown_connections 2023-10-06T04:32:38.068 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.063+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/1847519820 wait complete. 2023-10-06T04:32:38.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.064+0000 7f3ddf8b8700 1 Processor -- start 2023-10-06T04:32:38.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.064+0000 7f3ddf8b8700 1 -- start start 2023-10-06T04:32:38.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddf8b8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd8068bb0 0x7f3dd80678b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.071 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddf8b8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd80c7460 0x7f3dd8140010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddf8b8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd8155780 0x7f3dd8140550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddf8b8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd8140a90 con 0x7f3dd80c7460 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddf8b8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80c3840 con 0x7f3dd8155780 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddf8b8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dd80aab70 con 0x7f3dd8068bb0 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddce53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd80c7460 0x7f3dd8140010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddce53700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd80c7460 0x7f3dd8140010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48338/0 (socket says 172.21.15.123:48338) 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddce53700 1 -- 172.21.15.123:0/112446314 learned_addr learned my addr 172.21.15.123:0/112446314 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddd654700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd8068bb0 0x7f3dd80678b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.065+0000 7f3ddde55700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd8155780 0x7f3dd8140550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddce53700 1 -- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd8068bb0 msgr2=0x7f3dd80678b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddce53700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd8068bb0 0x7f3dd80678b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddce53700 1 -- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd8155780 msgr2=0x7f3dd8140550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddce53700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd8155780 0x7f3dd8140550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddce53700 1 -- 172.21.15.123:0/112446314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dd40094f0 con 0x7f3dd80c7460 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddde55700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd8155780 0x7f3dd8140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddce53700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd80c7460 0x7f3dd8140010 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c830 tx=0x7f3dc8008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3dcd7fa700 1 -- 172.21.15.123:0/112446314 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8011600 con 0x7f3dd80c7460 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3dcd7fa700 1 -- 172.21.15.123:0/112446314 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3dc8011760 con 0x7f3dd80c7460 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dd80aae50 con 0x7f3dd80c7460 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3dcd7fa700 1 -- 172.21.15.123:0/112446314 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3dc8011b20 con 0x7f3dd80c7460 2023-10-06T04:32:38.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.066+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dd80c3a90 con 0x7f3dd80c7460 2023-10-06T04:32:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.067+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dd806cc70 con 0x7f3dd80c7460 2023-10-06T04:32:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.067+0000 7f3dcd7fa700 1 -- 172.21.15.123:0/112446314 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3dc80193b0 con 0x7f3dd80c7460 2023-10-06T04:32:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.068+0000 7f3dcd7fa700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3dbc066f30 0x7f3dbc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.068+0000 7f3dcd7fa700 1 -- 172.21.15.123:0/112446314 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3dc8084900 con 0x7f3dd80c7460 2023-10-06T04:32:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.068+0000 7f3ddd654700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3dbc066f30 0x7f3dbc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.068+0000 7f3ddd654700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3dbc066f30 0x7f3dbc0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3dd806ea70 tx=0x7f3dd4016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:38.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.072+0000 7f3dcd7fa700 1 -- 172.21.15.123:0/112446314 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3dc80517f0 con 0x7f3dd80c7460 2023-10-06T04:32:38.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.253+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dd80c84f0 con 0x7f3dd80c7460 2023-10-06T04:32:38.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.254+0000 7f3dcd7fa700 1 -- 172.21.15.123:0/112446314 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v41) v1 ==== 74+0+18783 (secure 0 0 0) 0x7f3dc8054e10 con 0x7f3dd80c7460 2023-10-06T04:32:38.257 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:38.257 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":41,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:37.297115+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3dbc066f30 msgr2=0x7f3dbc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3dbc066f30 0x7f3dbc0693e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f3dd806ea70 tx=0x7f3dd4016000 comp rx=0 tx=0).stop 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd80c7460 msgr2=0x7f3dd8140010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd80c7460 0x7f3dd8140010 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f3dc800c830 tx=0x7f3dc8008040 comp rx=0 tx=0).stop 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 shutdown_connections 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3dbc066f30 0x7f3dbc0693e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3dd8155780 0x7f3dd8140550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3dd8068bb0 0x7f3dd80678b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 --2- 172.21.15.123:0/112446314 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3dd80c7460 0x7f3dd8140010 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 >> 172.21.15.123:0/112446314 conn(0x7f3dd80c9c90 msgr2=0x7f3dd806bb80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:38.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 shutdown_connections 2023-10-06T04:32:38.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.256+0000 7f3ddf8b8700 1 -- 172.21.15.123:0/112446314 wait complete. 2023-10-06T04:32:38.271 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs standby_count_wanted 0 2023-10-06T04:32:38.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.388+0000 7ff157ad4700 1 Processor -- start 2023-10-06T04:32:38.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.388+0000 7ff157ad4700 1 -- start start 2023-10-06T04:32:38.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff157ad4700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff1500c8340 0x7ff1500a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff157ad4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff1500c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff157ad4700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff150069aa0 con 0x7ff150068670 2023-10-06T04:32:38.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff157ad4700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff15006a340 con 0x7ff1500c8340 2023-10-06T04:32:38.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff157ad4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1500aa6f0 con 0x7ff1500c8710 2023-10-06T04:32:38.392 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff155870700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff1500c8340 0x7ff1500a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff15506f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff1500c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff155870700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff1500c8340 0x7ff1500a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:43768/0 (socket says 172.21.15.123:43768) 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff155870700 1 -- 172.21.15.123:0/2412476649 learned_addr learned my addr 172.21.15.123:0/2412476649 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff15506f700 1 -- 172.21.15.123:0/2412476649 >> v2:172.21.15.177:3300/0 conn(0x7ff1500c8340 msgr2=0x7ff1500a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff15506f700 1 --2- 172.21.15.123:0/2412476649 >> v2:172.21.15.177:3300/0 conn(0x7ff1500c8340 0x7ff1500a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff15506f700 1 -- 172.21.15.123:0/2412476649 >> v1:172.21.15.177:6789/0 conn(0x7ff150068670 legacy=0x7ff1500c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.389+0000 7ff15506f700 1 -- 172.21.15.123:0/2412476649 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff150069cf0 con 0x7ff1500c8710 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff15506f700 1 --2- 172.21.15.123:0/2412476649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff1500c8b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff140009910 tx=0x7ff1400091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8d7f68871953a8b5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff15486e700 1 -- 172.21.15.123:0/2412476649 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000b640 con 0x7ff1500c8710 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff15486e700 1 -- 172.21.15.123:0/2412476649 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14000b7a0 con 0x7ff1500c8710 2023-10-06T04:32:38.393 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff15486e700 1 -- 172.21.15.123:0/2412476649 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14000bb60 con 0x7ff1500c8710 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff157ad4700 1 -- 172.21.15.123:0/2412476649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 msgr2=0x7ff1500c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2412476649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff1500c8b60 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7ff140009910 tx=0x7ff1400091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff157ad4700 1 -- 172.21.15.123:0/2412476649 shutdown_connections 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2412476649 >> v2:172.21.15.177:3300/0 conn(0x7ff1500c8340 0x7ff1500a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2412476649 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff1500c8b60 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.390+0000 7ff157ad4700 1 -- 172.21.15.123:0/2412476649 >> 172.21.15.123:0/2412476649 conn(0x7ff1500c9ca0 msgr2=0x7ff15006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.391+0000 7ff157ad4700 1 -- 172.21.15.123:0/2412476649 shutdown_connections 2023-10-06T04:32:38.396 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.391+0000 7ff157ad4700 1 -- 172.21.15.123:0/2412476649 wait complete. 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 Processor -- start 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 -- start start 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff150068670 0x7ff15015b0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1500c8340 0x7ff15016a570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff150157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15015ac10 con 0x7ff150068670 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff15015a000 con 0x7ff1500c8340 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff157ad4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff15015a2e0 con 0x7ff1500c8710 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.392+0000 7ff155870700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff150068670 0x7ff15015b0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff150068670 0x7ff15015b0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48344/0 (socket says 172.21.15.123:48344) 2023-10-06T04:32:38.397 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 -- 172.21.15.123:0/2313193310 learned_addr learned my addr 172.21.15.123:0/2313193310 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff15506f700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1500c8340 0x7ff15016a570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff156071700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff150157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 -- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 msgr2=0x7ff150157890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff150157890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 -- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1500c8340 msgr2=0x7ff15016a570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1500c8340 0x7ff15016a570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:38.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 -- 172.21.15.123:0/2313193310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff140009420 con 0x7ff150068670 2023-10-06T04:32:38.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff155870700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff150068670 0x7ff15015b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00ea50 tx=0x7ff14c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:38.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff145ffb700 1 -- 172.21.15.123:0/2313193310 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00c850 con 0x7ff150068670 2023-10-06T04:32:38.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff145ffb700 1 -- 172.21.15.123:0/2313193310 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff14c00c9b0 con 0x7ff150068670 2023-10-06T04:32:38.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff145ffb700 1 -- 172.21.15.123:0/2313193310 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff14c00cd30 con 0x7ff150068670 2023-10-06T04:32:38.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff15015a5c0 con 0x7ff150068670 2023-10-06T04:32:38.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.393+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff150157dd0 con 0x7ff150068670 2023-10-06T04:32:38.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.394+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff150069fd0 con 0x7ff150068670 2023-10-06T04:32:38.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.395+0000 7ff145ffb700 1 -- 172.21.15.123:0/2313193310 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff14c013680 con 0x7ff150068670 2023-10-06T04:32:38.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.395+0000 7ff145ffb700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff134066f30 0x7ff1340693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:38.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.395+0000 7ff145ffb700 1 -- 172.21.15.123:0/2313193310 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff14c084b70 con 0x7ff150068670 2023-10-06T04:32:38.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.395+0000 7ff15506f700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff134066f30 0x7ff1340693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:38.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.396+0000 7ff15506f700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff134066f30 0x7ff1340693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1400091e0 tx=0x7ff140017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:38.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.399+0000 7ff145ffb700 1 -- 172.21.15.123:0/2313193310 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff14c051a60 con 0x7ff150068670 2023-10-06T04:32:38.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:38.625+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff15015aef0 con 0x7ff150068670 2023-10-06T04:32:39.249 INFO:tasks.ceph.mon.a.smithi123.stderr:2023-10-06T04:32:39.247+0000 7fea93165700 -1 log_channel(cluster) log [ERR] : Health check update: 2 filesystems are offline (MDS_ALL_DOWN) 2023-10-06T04:32:39.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.608+0000 7ff145ffb700 1 -- 172.21.15.123:0/2313193310 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "standby_count_wanted", "val": "0"}]=0 v21) v1 ==== 123+0+0 (secure 0 0 0) 0x7ff14c055080 con 0x7ff150068670 2023-10-06T04:32:39.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff134066f30 msgr2=0x7ff1340693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff134066f30 0x7ff1340693e0 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7ff1400091e0 tx=0x7ff140017000 comp rx=0 tx=0).stop 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff150068670 msgr2=0x7ff15015b0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff150068670 0x7ff15015b0a0 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7ff14c00ea50 tx=0x7ff14c009250 comp rx=0 tx=0).stop 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 shutdown_connections 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff134066f30 0x7ff1340693e0 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff150068670 0x7ff15015b0a0 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff1500c8710 0x7ff150157890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 --2- 172.21.15.123:0/2313193310 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff1500c8340 0x7ff15016a570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 >> 172.21.15.123:0/2313193310 conn(0x7ff1500c9ca0 msgr2=0x7ff150158a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:39.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 shutdown_connections 2023-10-06T04:32:39.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.611+0000 7ff157ad4700 1 -- 172.21.15.123:0/2313193310 wait complete. 2023-10-06T04:32:39.627 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs allow_standby_replay false 2023-10-06T04:32:39.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.752+0000 7f97ceea5700 1 Processor -- start 2023-10-06T04:32:39.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.753+0000 7f97ceea5700 1 -- start start 2023-10-06T04:32:39.757 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.753+0000 7f97ceea5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f97c8159650 0x7f97c80cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.753+0000 7f97ceea5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.753+0000 7f97ceea5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97c80cef40 0x7f97c80c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.753+0000 7f97ceea5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c80cf3a0 con 0x7f97c80cef40 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.753+0000 7f97ceea5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97c80cc850 con 0x7f97c8159650 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.753+0000 7f97ceea5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80cbc40 con 0x7f97c80cb630 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97c80cef40 0x7f97c80c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f97c80cef40 0x7f97c80c5250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48356/0 (socket says 172.21.15.123:48356) 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 -- 172.21.15.123:0/3900335648 learned_addr learned my addr 172.21.15.123:0/3900335648 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97ccc41700 1 --2- 172.21.15.123:0/3900335648 >> v2:172.21.15.177:3300/0 conn(0x7f97c8159650 0x7f97c80cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97c7fff700 1 --2- 172.21.15.123:0/3900335648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:39.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 -- 172.21.15.123:0/3900335648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 msgr2=0x7f97c80cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 --2- 172.21.15.123:0/3900335648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 -- 172.21.15.123:0/3900335648 >> v2:172.21.15.177:3300/0 conn(0x7f97c8159650 msgr2=0x7f97c80cb0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 --2- 172.21.15.123:0/3900335648 >> v2:172.21.15.177:3300/0 conn(0x7f97c8159650 0x7f97c80cb0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 -- 172.21.15.123:0/3900335648 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c80cbf20 con 0x7f97c80cef40 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97ccc41700 1 --2- 172.21.15.123:0/3900335648 >> v2:172.21.15.177:3300/0 conn(0x7f97c8159650 0x7f97c80cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97c7fff700 1 --2- 172.21.15.123:0/3900335648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97cd442700 1 --2- 172.21.15.123:0/3900335648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c80c5250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f97c8099d40 tx=0x7f97c0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462c90bd16f45f9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97c77fe700 1 -- 172.21.15.123:0/3900335648 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c000e070 con 0x7f97c80cef40 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97c77fe700 1 -- 172.21.15.123:0/3900335648 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c000ba50 con 0x7f97c80cef40 2023-10-06T04:32:39.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.754+0000 7f97c77fe700 1 -- 172.21.15.123:0/3900335648 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00092e0 con 0x7f97c80cef40 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 -- 172.21.15.123:0/3900335648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 msgr2=0x7f97c80c5250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 --2- 172.21.15.123:0/3900335648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c80c5250 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f97c8099d40 tx=0x7f97c0009840 comp rx=0 tx=0).stop 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 -- 172.21.15.123:0/3900335648 shutdown_connections 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 --2- 172.21.15.123:0/3900335648 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c80c5250 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 --2- 172.21.15.123:0/3900335648 >> v2:172.21.15.177:3300/0 conn(0x7f97c8159650 0x7f97c80cb0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 --2- 172.21.15.123:0/3900335648 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 -- 172.21.15.123:0/3900335648 >> 172.21.15.123:0/3900335648 conn(0x7f97c8099610 msgr2=0x7f97c8063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 -- 172.21.15.123:0/3900335648 shutdown_connections 2023-10-06T04:32:39.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.755+0000 7f97ceea5700 1 -- 172.21.15.123:0/3900335648 wait complete. 2023-10-06T04:32:39.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.756+0000 7f97ceea5700 1 Processor -- start 2023-10-06T04:32:39.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.756+0000 7f97ceea5700 1 -- start start 2023-10-06T04:32:39.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ceea5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80ca3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:39.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ceea5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c8144ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:39.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ceea5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97c8159650 0x7f97c81696e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:39.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ceea5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c7a10 con 0x7f97c80cef40 2023-10-06T04:32:39.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ceea5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97c8157980 con 0x7f97c8159650 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ceea5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97c80c7460 con 0x7f97c80cb630 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c8144ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c8144ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48358/0 (socket says 172.21.15.123:48358) 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 -- 172.21.15.123:0/4050430228 learned_addr learned my addr 172.21.15.123:0/4050430228 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ccc41700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80ca3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97cd442700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97c8159650 0x7f97c81696e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 -- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 msgr2=0x7f97c80ca3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80ca3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 -- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97c8159650 msgr2=0x7f97c81696e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97c8159650 0x7f97c81696e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 -- 172.21.15.123:0/4050430228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c0009ae0 con 0x7f97c80cef40 2023-10-06T04:32:39.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c7fff700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c8144ef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f97b400d860 tx=0x7f97b4008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c4ff9700 1 -- 172.21.15.123:0/4050430228 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b40125f0 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c4ff9700 1 -- 172.21.15.123:0/4050430228 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97b4012750 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97c4ff9700 1 -- 172.21.15.123:0/4050430228 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97b4012ad0 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.757+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97c80c7730 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.758+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97c8157610 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.758+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97c8157b10 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.759+0000 7f97c4ff9700 1 -- 172.21.15.123:0/4050430228 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f97b4014430 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.759+0000 7f97c4ff9700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f97ac066f30 0x7f97ac0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.759+0000 7f97c4ff9700 1 -- 172.21.15.123:0/4050430228 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f97b4084d30 con 0x7f97c80cef40 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.759+0000 7f97ccc41700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f97ac066f30 0x7f97ac0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:39.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.760+0000 7f97ccc41700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f97ac066f30 0x7f97ac0693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009a80 tx=0x7f97bc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:39.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.763+0000 7f97c4ff9700 1 -- 172.21.15.123:0/4050430228 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f97b40556b0 con 0x7f97c80cef40 2023-10-06T04:32:39.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:39.987+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f97c80cc290 con 0x7f97c80cef40 2023-10-06T04:32:40.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.610+0000 7f97c4ff9700 1 -- 172.21.15.123:0/4050430228 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "allow_standby_replay", "val": "false"}]=0 v22) v1 ==== 127+0+0 (secure 0 0 0) 0x7f97b4055240 con 0x7f97c80cef40 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f97ac066f30 msgr2=0x7f97ac0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f97ac066f30 0x7f97ac0693e0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f97bc009a80 tx=0x7f97bc008040 comp rx=0 tx=0).stop 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 msgr2=0x7f97c8144ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c8144ef0 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f97b400d860 tx=0x7f97b4008040 comp rx=0 tx=0).stop 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 shutdown_connections 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f97ac066f30 0x7f97ac0693e0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f97c8159650 0x7f97c81696e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f97c80cb630 0x7f97c80ca3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 --2- 172.21.15.123:0/4050430228 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f97c80cef40 0x7f97c8144ef0 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 >> 172.21.15.123:0/4050430228 conn(0x7f97c8099610 msgr2=0x7f97c8069260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:40.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 shutdown_connections 2023-10-06T04:32:40.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.612+0000 7f97ceea5700 1 -- 172.21.15.123:0/4050430228 wait complete. 2023-10-06T04:32:40.629 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:40.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.749+0000 7ff2bb962700 1 Processor -- start 2023-10-06T04:32:40.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.749+0000 7ff2bb962700 1 -- start start 2023-10-06T04:32:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2bb962700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff2b4156f60 0x7ff2b415ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:40.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2bb962700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b415b290 0x7ff2b40ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:40.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2bb962700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2b4157910 con 0x7ff2b406cd60 2023-10-06T04:32:40.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2bb962700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2b406d240 con 0x7ff2b4156f60 2023-10-06T04:32:40.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2bb962700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b406d520 con 0x7ff2b415b290 2023-10-06T04:32:40.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2b8efd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b415b290 0x7ff2b40ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:40.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2b96fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff2b4156f60 0x7ff2b415ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:40.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2b96fe700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff2b4156f60 0x7ff2b415ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:55870/0 (socket says 172.21.15.123:55870) 2023-10-06T04:32:40.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2b8efd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b415b290 0x7ff2b40ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:52722/0 (socket says 172.21.15.123:52722) 2023-10-06T04:32:40.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.750+0000 7ff2b96fe700 1 -- 172.21.15.123:0/21236390 learned_addr learned my addr 172.21.15.123:0/21236390 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:40.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.751+0000 7ff2b8efd700 1 -- 172.21.15.123:0/21236390 >> v2:172.21.15.177:3300/0 conn(0x7ff2b4156f60 msgr2=0x7ff2b415ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:40.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.751+0000 7ff2b8efd700 1 --2- 172.21.15.123:0/21236390 >> v2:172.21.15.177:3300/0 conn(0x7ff2b4156f60 0x7ff2b415ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.751+0000 7ff2b8efd700 1 -- 172.21.15.123:0/21236390 >> v1:172.21.15.177:6789/0 conn(0x7ff2b406cd60 legacy=0x7ff2b4157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.751+0000 7ff2b8efd700 1 -- 172.21.15.123:0/21236390 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b406d800 con 0x7ff2b415b290 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.751+0000 7ff2b96fe700 1 --2- 172.21.15.123:0/21236390 >> v2:172.21.15.177:3300/0 conn(0x7ff2b4156f60 0x7ff2b415ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.751+0000 7ff2b8efd700 1 --2- 172.21.15.123:0/21236390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b415b290 0x7ff2b40ad730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40098e0 tx=0x7ff2a40091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da85c52a1e932e6d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.751+0000 7ff2abfff700 1 -- 172.21.15.123:0/21236390 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400b640 con 0x7ff2b415b290 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2abfff700 1 -- 172.21.15.123:0/21236390 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2a400b7a0 con 0x7ff2b415b290 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2abfff700 1 -- 172.21.15.123:0/21236390 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2a400bb40 con 0x7ff2b415b290 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 -- 172.21.15.123:0/21236390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b415b290 msgr2=0x7ff2b40ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 --2- 172.21.15.123:0/21236390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b415b290 0x7ff2b40ad730 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7ff2a40098e0 tx=0x7ff2a40091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 -- 172.21.15.123:0/21236390 shutdown_connections 2023-10-06T04:32:40.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 --2- 172.21.15.123:0/21236390 >> v2:172.21.15.177:3300/0 conn(0x7ff2b4156f60 0x7ff2b415ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 --2- 172.21.15.123:0/21236390 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b415b290 0x7ff2b40ad730 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 -- 172.21.15.123:0/21236390 >> 172.21.15.123:0/21236390 conn(0x7ff2b40ce0c0 msgr2=0x7ff2b40ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 -- 172.21.15.123:0/21236390 shutdown_connections 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.752+0000 7ff2bb962700 1 -- 172.21.15.123:0/21236390 wait complete. 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.753+0000 7ff2bb962700 1 Processor -- start 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.753+0000 7ff2bb962700 1 -- start start 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2bb962700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2b406cd60 0x7ff2b415c140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2bb962700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b4156f60 0x7ff2b40c4360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2bb962700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2b415b290 0x7ff2b414b650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2bb962700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b415c680 con 0x7ff2b406cd60 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2bb962700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b40c48a0 con 0x7ff2b415b290 2023-10-06T04:32:40.764 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2bb962700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2b414bb90 con 0x7ff2b4156f60 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2b96fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2b406cd60 0x7ff2b415c140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2b96fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2b406cd60 0x7ff2b415c140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33552/0 (socket says 172.21.15.123:33552) 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2b96fe700 1 -- 172.21.15.123:0/1663392418 learned_addr learned my addr 172.21.15.123:0/1663392418 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2b8efd700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b4156f60 0x7ff2b40c4360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.754+0000 7ff2b9eff700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2b415b290 0x7ff2b414b650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b96fe700 1 -- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b4156f60 msgr2=0x7ff2b40c4360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b96fe700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b4156f60 0x7ff2b40c4360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b96fe700 1 -- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2b415b290 msgr2=0x7ff2b414b650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b96fe700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2b415b290 0x7ff2b414b650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b96fe700 1 -- 172.21.15.123:0/1663392418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b00094f0 con 0x7ff2b406cd60 2023-10-06T04:32:40.765 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b8efd700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b4156f60 0x7ff2b40c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b9eff700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2b415b290 0x7ff2b414b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2b96fe700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2b406cd60 0x7ff2b415c140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000ec50 tx=0x7ff2b0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2a97fa700 1 -- 172.21.15.123:0/1663392418 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b000c9d0 con 0x7ff2b406cd60 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2a97fa700 1 -- 172.21.15.123:0/1663392418 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b000cb30 con 0x7ff2b406cd60 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2a4009420 con 0x7ff2b406cd60 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2a97fa700 1 -- 172.21.15.123:0/1663392418 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0013470 con 0x7ff2b406cd60 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.755+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2b416b370 con 0x7ff2b406cd60 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.756+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2b414bd50 con 0x7ff2b406cd60 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.756+0000 7ff2a97fa700 1 -- 172.21.15.123:0/1663392418 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff2b0013860 con 0x7ff2b406cd60 2023-10-06T04:32:40.766 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.756+0000 7ff2a97fa700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff298066f30 0x7ff2980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:40.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.757+0000 7ff2a97fa700 1 -- 172.21.15.123:0/1663392418 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7ff2b00850a0 con 0x7ff2b406cd60 2023-10-06T04:32:40.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.757+0000 7ff2b8efd700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff298066f30 0x7ff2980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:40.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.757+0000 7ff2b8efd700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff298066f30 0x7ff2980693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2b415b690 tx=0x7ff2a4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:40.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.761+0000 7ff2a97fa700 1 -- 172.21.15.123:0/1663392418 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff2b0051f90 con 0x7ff2b406cd60 2023-10-06T04:32:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.985+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2b406dae0 con 0x7ff2b406cd60 2023-10-06T04:32:40.987 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.985+0000 7ff2a97fa700 1 -- 172.21.15.123:0/1663392418 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 22 v22) v1 ==== 94+0+5663 (secure 0 0 0) 0x7ff2b00555b0 con 0x7ff2b406cd60 2023-10-06T04:32:40.988 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:40.988 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":22,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5295,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":22,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:40.608007+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:32:40.989 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff298066f30 msgr2=0x7ff2980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff298066f30 0x7ff2980693e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7ff2b415b690 tx=0x7ff2a4017000 comp rx=0 tx=0).stop 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2b406cd60 msgr2=0x7ff2b415c140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2b406cd60 0x7ff2b415c140 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7ff2b000ec50 tx=0x7ff2b0009250 comp rx=0 tx=0).stop 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 shutdown_connections 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff298066f30 0x7ff2980693e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff2b406cd60 0x7ff2b415c140 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff2b415b290 0x7ff2b414b650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 --2- 172.21.15.123:0/1663392418 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff2b4156f60 0x7ff2b40c4360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 >> 172.21.15.123:0/1663392418 conn(0x7ff2b40ce0c0 msgr2=0x7ff2b40b1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:40.990 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 shutdown_connections 2023-10-06T04:32:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:40.987+0000 7ff2bb962700 1 -- 172.21.15.123:0/1663392418 wait complete. 2023-10-06T04:32:40.991 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 22 2023-10-06T04:32:41.002 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:41.119 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.117+0000 7fe882015700 1 Processor -- start 2023-10-06T04:32:41.120 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.117+0000 7fe882015700 1 -- start start 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe882015700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe87c06e960 0x7fe87c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe882015700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06f080 0x7fe87c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe882015700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe87c0c8580 con 0x7fe87c1560c0 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe882015700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe87c0c8860 con 0x7fe87c06e960 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe882015700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0c8b40 con 0x7fe87c06f080 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87affd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe87c06e960 0x7fe87c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87b7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe87c1560c0 0x7fe87c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45036/0 (socket says 172.21.15.123:45036) 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87b7fe700 1 -- 172.21.15.123:0/3395581900 learned_addr learned my addr 172.21.15.123:0/3395581900 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:41.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87bfff700 1 --2- 172.21.15.123:0/3395581900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06f080 0x7fe87c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87affd700 1 -- 172.21.15.123:0/3395581900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06f080 msgr2=0x7fe87c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87affd700 1 --2- 172.21.15.123:0/3395581900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06f080 0x7fe87c0ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87affd700 1 -- 172.21.15.123:0/3395581900 >> v1:172.21.15.123:6789/0 conn(0x7fe87c1560c0 legacy=0x7fe87c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87a7fc700 1 -- 172.21.15.123:0/3395581900 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 561057620 0 0) 0x7fe87c0c8580 con 0x7fe87c1560c0 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87affd700 1 -- 172.21.15.123:0/3395581900 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe87c0c8e20 con 0x7fe87c06e960 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.118+0000 7fe87affd700 1 --2- 172.21.15.123:0/3395581900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06e960 0x7fe87c0c7fa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0098e0 tx=0x7fe86c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc6c56528797604 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe87a7fc700 1 -- 172.21.15.123:0/3395581900 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c009e80 con 0x7fe87c06e960 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe87a7fc700 1 -- 172.21.15.123:0/3395581900 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c00b860 con 0x7fe87c06e960 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe87a7fc700 1 -- 172.21.15.123:0/3395581900 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00bbe0 con 0x7fe87c06e960 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 -- 172.21.15.123:0/3395581900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06e960 msgr2=0x7fe87c0c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 --2- 172.21.15.123:0/3395581900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06e960 0x7fe87c0c7fa0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7fe86c0098e0 tx=0x7fe86c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 -- 172.21.15.123:0/3395581900 shutdown_connections 2023-10-06T04:32:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 --2- 172.21.15.123:0/3395581900 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06f080 0x7fe87c0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 --2- 172.21.15.123:0/3395581900 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06e960 0x7fe87c0c7fa0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 -- 172.21.15.123:0/3395581900 >> 172.21.15.123:0/3395581900 conn(0x7fe87c0cda80 msgr2=0x7fe87c0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 -- 172.21.15.123:0/3395581900 shutdown_connections 2023-10-06T04:32:41.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.119+0000 7fe882015700 1 -- 172.21.15.123:0/3395581900 wait complete. 2023-10-06T04:32:41.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.120+0000 7fe882015700 1 Processor -- start 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.120+0000 7fe882015700 1 -- start start 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe882015700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06e960 0x7fe87c156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe882015700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06f080 0x7fe87c0b2200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe882015700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87c1560c0 0x7fe87c16b290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe882015700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c157430 con 0x7fe87c06f080 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe882015700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c0c5610 con 0x7fe87c1560c0 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe882015700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe87c156b90 con 0x7fe87c06e960 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe87affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06f080 0x7fe87c0b2200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe87affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06f080 0x7fe87c0b2200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33558/0 (socket says 172.21.15.123:33558) 2023-10-06T04:32:41.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe87affd700 1 -- 172.21.15.123:0/2839632940 learned_addr learned my addr 172.21.15.123:0/2839632940 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe87b7fe700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06e960 0x7fe87c156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.121+0000 7fe87bfff700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87c1560c0 0x7fe87c16b290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe87affd700 1 -- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06e960 msgr2=0x7fe87c156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe87affd700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06e960 0x7fe87c156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe87affd700 1 -- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87c1560c0 msgr2=0x7fe87c16b290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe87affd700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87c1560c0 0x7fe87c16b290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe87affd700 1 -- 172.21.15.123:0/2839632940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe86c009420 con 0x7fe87c06f080 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe87bfff700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87c1560c0 0x7fe87c16b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe87affd700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06f080 0x7fe87c0b2200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe86c013000 tx=0x7fe86c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe8637fe700 1 -- 172.21.15.123:0/2839632940 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c00bbe0 con 0x7fe87c06f080 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe8637fe700 1 -- 172.21.15.123:0/2839632940 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe86c011420 con 0x7fe87c06f080 2023-10-06T04:32:41.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe87c156e70 con 0x7fe87c06f080 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe8637fe700 1 -- 172.21.15.123:0/2839632940 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe86c0117a0 con 0x7fe87c06f080 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.122+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe87c16ad70 con 0x7fe87c06f080 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.123+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe87c16abd0 con 0x7fe87c06f080 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.123+0000 7fe8637fe700 1 -- 172.21.15.123:0/2839632940 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe86c011b90 con 0x7fe87c06f080 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.123+0000 7fe8637fe700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe868066fb0 0x7fe868069460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.124+0000 7fe8637fe700 1 -- 172.21.15.123:0/2839632940 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe86c085e60 con 0x7fe87c06f080 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.124+0000 7fe87b7fe700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe868066fb0 0x7fe868069460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.124+0000 7fe87b7fe700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe868066fb0 0x7fe868069460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe87000c870 tx=0x7fe870009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.128+0000 7fe8637fe700 1 -- 172.21.15.123:0/2839632940 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe87c16abd0 con 0x7fe87c06f080 2023-10-06T04:32:41.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.310+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe87c16abd0 con 0x7fe87c06f080 2023-10-06T04:32:41.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.311+0000 7fe8637fe700 1 -- 172.21.15.123:0/2839632940 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18770 (secure 0 0 0) 0x7fe87c16abd0 con 0x7fe87c06f080 2023-10-06T04:32:41.314 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:41.314 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:38.586940+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe868066fb0 msgr2=0x7fe868069460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe868066fb0 0x7fe868069460 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fe87000c870 tx=0x7fe870009040 comp rx=0 tx=0).stop 2023-10-06T04:32:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06f080 msgr2=0x7fe87c0b2200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06f080 0x7fe87c0b2200 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7fe86c013000 tx=0x7fe86c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:41.316 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 shutdown_connections 2023-10-06T04:32:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe868066fb0 0x7fe868069460 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe87c1560c0 0x7fe87c16b290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe87c06e960 0x7fe87c156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 --2- 172.21.15.123:0/2839632940 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe87c06f080 0x7fe87c0b2200 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 >> 172.21.15.123:0/2839632940 conn(0x7fe87c0cda80 msgr2=0x7fe87c0ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 shutdown_connections 2023-10-06T04:32:41.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.313+0000 7fe882015700 1 -- 172.21.15.123:0/2839632940 wait complete. 2023-10-06T04:32:41.328 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-10-06T04:32:41.328 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:32:41.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.443+0000 7f3cb7450700 1 Processor -- start 2023-10-06T04:32:41.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.443+0000 7f3cb7450700 1 -- start start 2023-10-06T04:32:41.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb7450700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f3cb00688c0 0x7f3cb006a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb7450700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb006a930 0x7f3cb0156de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb7450700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3cb0068c90 0x7f3cb0157320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb7450700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00cc620 con 0x7f3cb0068c90 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb7450700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cb00cc190 con 0x7f3cb00688c0 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb7450700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb0157860 con 0x7f3cb006a930 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3cb0068c90 0x7f3cb0157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f3cb0068c90 0x7f3cb0157320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33574/0 (socket says 172.21.15.123:33574) 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 -- 172.21.15.123:0/2803165385 learned_addr learned my addr 172.21.15.123:0/2803165385 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb49eb700 1 --2- 172.21.15.123:0/2803165385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb006a930 0x7f3cb0156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb51ec700 1 --2- 172.21.15.123:0/2803165385 >> v2:172.21.15.177:3300/0 conn(0x7f3cb00688c0 0x7f3cb006a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 -- 172.21.15.123:0/2803165385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb006a930 msgr2=0x7f3cb0156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 --2- 172.21.15.123:0/2803165385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb006a930 0x7f3cb0156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 -- 172.21.15.123:0/2803165385 >> v2:172.21.15.177:3300/0 conn(0x7f3cb00688c0 msgr2=0x7f3cb006a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 --2- 172.21.15.123:0/2803165385 >> v2:172.21.15.177:3300/0 conn(0x7f3cb00688c0 0x7f3cb006a3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 -- 172.21.15.123:0/2803165385 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cb00cb580 con 0x7f3cb0068c90 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3cb59ed700 1 --2- 172.21.15.123:0/2803165385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb0068c90 0x7f3cb0157320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00698e0 tx=0x7f3ca8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8a16598f5a773cd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.444+0000 7f3ca7fff700 1 -- 172.21.15.123:0/2803165385 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca800e070 con 0x7f3cb0068c90 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3ca7fff700 1 -- 172.21.15.123:0/2803165385 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca800b970 con 0x7f3cb0068c90 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3ca7fff700 1 -- 172.21.15.123:0/2803165385 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca80092e0 con 0x7f3cb0068c90 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 -- 172.21.15.123:0/2803165385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb0068c90 msgr2=0x7f3cb0157320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 --2- 172.21.15.123:0/2803165385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb0068c90 0x7f3cb0157320 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f3cb00698e0 tx=0x7f3ca8009840 comp rx=0 tx=0).stop 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 -- 172.21.15.123:0/2803165385 shutdown_connections 2023-10-06T04:32:41.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 --2- 172.21.15.123:0/2803165385 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb0068c90 0x7f3cb0157320 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 --2- 172.21.15.123:0/2803165385 >> v2:172.21.15.177:3300/0 conn(0x7f3cb00688c0 0x7f3cb006a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 --2- 172.21.15.123:0/2803165385 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb006a930 0x7f3cb0156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 -- 172.21.15.123:0/2803165385 >> 172.21.15.123:0/2803165385 conn(0x7f3cb00c9c90 msgr2=0x7f3cb006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:41.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 -- 172.21.15.123:0/2803165385 shutdown_connections 2023-10-06T04:32:41.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.445+0000 7f3cb7450700 1 -- 172.21.15.123:0/2803165385 wait complete. 2023-10-06T04:32:41.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.447+0000 7f3cb7450700 1 Processor -- start 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.447+0000 7f3cb7450700 1 -- start start 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb7450700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb00688c0 0x7f3cb00b0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb7450700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb0068c90 0x7f3cb01465f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb7450700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cb006a930 0x7f3cb00b1e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb7450700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00b05a0 con 0x7f3cb00688c0 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb7450700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00b0db0 con 0x7f3cb006a930 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb7450700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cb00b1090 con 0x7f3cb0068c90 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb00688c0 0x7f3cb00b0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb00688c0 0x7f3cb00b0060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33582/0 (socket says 172.21.15.123:33582) 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 -- 172.21.15.123:0/1261068584 learned_addr learned my addr 172.21.15.123:0/1261068584 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:41.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 -- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb0068c90 msgr2=0x7f3cb01465f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb59ed700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cb006a930 0x7f3cb00b1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb49eb700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb0068c90 0x7f3cb01465f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb0068c90 0x7f3cb01465f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 -- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cb006a930 msgr2=0x7f3cb00b1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cb006a930 0x7f3cb00b1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 -- 172.21.15.123:0/1261068584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca8009ae0 con 0x7f3cb00688c0 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.448+0000 7f3cb51ec700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb00688c0 0x7f3cb00b0060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000ec80 tx=0x7f3ca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.449+0000 7f3ca57fa700 1 -- 172.21.15.123:0/1261068584 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca000c9d0 con 0x7f3cb00688c0 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.449+0000 7f3ca57fa700 1 -- 172.21.15.123:0/1261068584 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca000cb30 con 0x7f3cb00688c0 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.449+0000 7f3ca57fa700 1 -- 172.21.15.123:0/1261068584 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca0013470 con 0x7f3cb00688c0 2023-10-06T04:32:41.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.449+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cb00b1370 con 0x7f3cb00688c0 2023-10-06T04:32:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.449+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cb00b1930 con 0x7f3cb00688c0 2023-10-06T04:32:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.450+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cb006cc70 con 0x7f3cb00688c0 2023-10-06T04:32:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.450+0000 7f3ca57fa700 1 -- 172.21.15.123:0/1261068584 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3ca0013820 con 0x7f3cb00688c0 2023-10-06T04:32:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.450+0000 7f3ca57fa700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3c94066f30 0x7f3c940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.450+0000 7f3ca57fa700 1 -- 172.21.15.123:0/1261068584 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3ca0084cc0 con 0x7f3cb00688c0 2023-10-06T04:32:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.450+0000 7f3cb49eb700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3c94066f30 0x7f3c940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.451+0000 7f3cb49eb700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3c94066f30 0x7f3c940693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3cac006f00 tx=0x7f3cac008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.455+0000 7f3ca57fa700 1 -- 172.21.15.123:0/1261068584 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3ca0051bb0 con 0x7f3cb00688c0 2023-10-06T04:32:41.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.635+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3cb00cb860 con 0x7f3c94066f30 2023-10-06T04:32:41.647 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.645+0000 7f3ca57fa700 1 -- 172.21.15.123:0/1261068584 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+386486 (secure 0 0 0) 0x7f3cb00cb860 con 0x7f3c94066f30 2023-10-06T04:32:41.652 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:41.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3c94066f30 msgr2=0x7f3c940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3c94066f30 0x7f3c940693e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f3cac006f00 tx=0x7f3cac008040 comp rx=0 tx=0).stop 2023-10-06T04:32:41.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb00688c0 msgr2=0x7f3cb00b0060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb00688c0 0x7f3cb00b0060 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f3ca000ec80 tx=0x7f3ca00091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 shutdown_connections 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3c94066f30 0x7f3c940693e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3cb00688c0 0x7f3cb00b0060 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3cb006a930 0x7f3cb00b1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 --2- 172.21.15.123:0/1261068584 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3cb0068c90 0x7f3cb01465f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.648+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 >> 172.21.15.123:0/1261068584 conn(0x7f3cb00c9c90 msgr2=0x7f3cb00ca060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.649+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 shutdown_connections 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.649+0000 7f3cb7450700 1 -- 172.21.15.123:0/1261068584 wait complete. 2023-10-06T04:32:41.656 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:32:41.663 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-10-06T04:32:40.902595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395196,"kb_used_data":16864,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752579524,"statfs":{"total":773094113280,"available":770641432576,"internally_reserved":0,"allocated":17268736,"data_stored":15231248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6289150,"num_objects":-20,"num_object_clones":0,"num_object_copies":-40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-20,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":51,"num_write_kb":6167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001036"},"pg_stats":[{"pgid":"6.3f","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270288+0000","last_change":"2023-10-06T04:32:24.268479+0000","last_active":"2023-10-06T04:32:34.270288+0000","last_peered":"2023-10-06T04:32:34.270288+0000","last_clean":"2023-10-06T04:32:34.270288+0000","last_became_active":"2023-10-06T04:32:24.268175+0000","last_became_peered":"2023-10-06T04:32:24.268175+0000","last_unstale":"2023-10-06T04:32:34.270288+0000","last_undegraded":"2023-10-06T04:32:34.270288+0000","last_fullsized":"2023-10-06T04:32:34.270288+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"34'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270143+0000","last_change":"2023-10-06T04:32:24.269331+0000","last_active":"2023-10-06T04:32:34.270143+0000","last_peered":"2023-10-06T04:32:34.270143+0000","last_clean":"2023-10-06T04:32:34.270143+0000","last_became_active":"2023-10-06T04:32:24.269096+0000","last_became_peered":"2023-10-06T04:32:24.269096+0000","last_unstale":"2023-10-06T04:32:34.270143+0000","last_undegraded":"2023-10-06T04:32:34.270143+0000","last_fullsized":"2023-10-06T04:32:34.270143+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270084+0000","last_change":"2023-10-06T04:32:24.267252+0000","last_active":"2023-10-06T04:32:34.270084+0000","last_peered":"2023-10-06T04:32:34.270084+0000","last_clean":"2023-10-06T04:32:34.270084+0000","last_became_active":"2023-10-06T04:32:24.266928+0000","last_became_peered":"2023-10-06T04:32:24.266928+0000","last_unstale":"2023-10-06T04:32:34.270084+0000","last_undegraded":"2023-10-06T04:32:34.270084+0000","last_fullsized":"2023-10-06T04:32:34.270084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263072+0000","last_change":"2023-10-06T04:32:24.271555+0000","last_active":"2023-10-06T04:32:34.263072+0000","last_peered":"2023-10-06T04:32:34.263072+0000","last_clean":"2023-10-06T04:32:34.263072+0000","last_became_active":"2023-10-06T04:32:24.271271+0000","last_became_peered":"2023-10-06T04:32:24.271271+0000","last_unstale":"2023-10-06T04:32:34.263072+0000","last_undegraded":"2023-10-06T04:32:34.263072+0000","last_fullsized":"2023-10-06T04:32:34.263072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"34'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263100+0000","last_change":"2023-10-06T04:32:24.270442+0000","last_active":"2023-10-06T04:32:34.263100+0000","last_peered":"2023-10-06T04:32:34.263100+0000","last_clean":"2023-10-06T04:32:34.263100+0000","last_became_active":"2023-10-06T04:32:24.270131+0000","last_became_peered":"2023-10-06T04:32:24.270131+0000","last_unstale":"2023-10-06T04:32:34.263100+0000","last_undegraded":"2023-10-06T04:32:34.263100+0000","last_fullsized":"2023-10-06T04:32:34.263100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268374+0000","last_change":"2023-10-06T04:32:24.263272+0000","last_active":"2023-10-06T04:32:34.268374+0000","last_peered":"2023-10-06T04:32:34.268374+0000","last_clean":"2023-10-06T04:32:34.268374+0000","last_became_active":"2023-10-06T04:32:24.262984+0000","last_became_peered":"2023-10-06T04:32:24.262984+0000","last_unstale":"2023-10-06T04:32:34.268374+0000","last_undegraded":"2023-10-06T04:32:34.268374+0000","last_fullsized":"2023-10-06T04:32:34.268374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:48.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315314+0000","last_change":"2023-10-06T04:32:24.263913+0000","last_active":"2023-10-06T04:32:35.315314+0000","last_peered":"2023-10-06T04:32:35.315314+0000","last_clean":"2023-10-06T04:32:35.315314+0000","last_became_active":"2023-10-06T04:32:24.263358+0000","last_became_peered":"2023-10-06T04:32:24.263358+0000","last_unstale":"2023-10-06T04:32:35.315314+0000","last_undegraded":"2023-10-06T04:32:35.315314+0000","last_fullsized":"2023-10-06T04:32:35.315314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269238+0000","last_change":"2023-10-06T04:32:24.266865+0000","last_active":"2023-10-06T04:32:34.269238+0000","last_peered":"2023-10-06T04:32:34.269238+0000","last_clean":"2023-10-06T04:32:34.269238+0000","last_became_active":"2023-10-06T04:32:24.266456+0000","last_became_peered":"2023-10-06T04:32:24.266456+0000","last_unstale":"2023-10-06T04:32:34.269238+0000","last_undegraded":"2023-10-06T04:32:34.269238+0000","last_fullsized":"2023-10-06T04:32:34.269238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"34'11","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270525+0000","last_change":"2023-10-06T04:32:24.267567+0000","last_active":"2023-10-06T04:32:34.270525+0000","last_peered":"2023-10-06T04:32:34.270525+0000","last_clean":"2023-10-06T04:32:34.270525+0000","last_became_active":"2023-10-06T04:32:24.267326+0000","last_became_peered":"2023-10-06T04:32:24.267326+0000","last_unstale":"2023-10-06T04:32:34.270525+0000","last_undegraded":"2023-10-06T04:32:34.270525+0000","last_fullsized":"2023-10-06T04:32:34.270525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"34'15","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315604+0000","last_change":"2023-10-06T04:32:24.266439+0000","last_active":"2023-10-06T04:32:35.315604+0000","last_peered":"2023-10-06T04:32:35.315604+0000","last_clean":"2023-10-06T04:32:35.315604+0000","last_became_active":"2023-10-06T04:32:24.266172+0000","last_became_peered":"2023-10-06T04:32:24.266172+0000","last_unstale":"2023-10-06T04:32:35.315604+0000","last_undegraded":"2023-10-06T04:32:35.315604+0000","last_fullsized":"2023-10-06T04:32:35.315604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:07.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"34'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315594+0000","last_change":"2023-10-06T04:32:24.264976+0000","last_active":"2023-10-06T04:32:35.315594+0000","last_peered":"2023-10-06T04:32:35.315594+0000","last_clean":"2023-10-06T04:32:35.315594+0000","last_became_active":"2023-10-06T04:32:24.264647+0000","last_became_peered":"2023-10-06T04:32:24.264647+0000","last_unstale":"2023-10-06T04:32:35.315594+0000","last_undegraded":"2023-10-06T04:32:35.315594+0000","last_fullsized":"2023-10-06T04:32:35.315594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269897+0000","last_change":"2023-10-06T04:32:24.262121+0000","last_active":"2023-10-06T04:32:34.269897+0000","last_peered":"2023-10-06T04:32:34.269897+0000","last_clean":"2023-10-06T04:32:34.269897+0000","last_became_active":"2023-10-06T04:32:24.261846+0000","last_became_peered":"2023-10-06T04:32:24.261846+0000","last_unstale":"2023-10-06T04:32:34.269897+0000","last_undegraded":"2023-10-06T04:32:34.269897+0000","last_fullsized":"2023-10-06T04:32:34.269897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315289+0000","last_change":"2023-10-06T04:32:24.267116+0000","last_active":"2023-10-06T04:32:35.315289+0000","last_peered":"2023-10-06T04:32:35.315289+0000","last_clean":"2023-10-06T04:32:35.315289+0000","last_became_active":"2023-10-06T04:32:24.266756+0000","last_became_peered":"2023-10-06T04:32:24.266756+0000","last_unstale":"2023-10-06T04:32:35.315289+0000","last_undegraded":"2023-10-06T04:32:35.315289+0000","last_fullsized":"2023-10-06T04:32:35.315289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:26.457033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261300+0000","last_change":"2023-10-06T04:32:24.267554+0000","last_active":"2023-10-06T04:32:34.261300+0000","last_peered":"2023-10-06T04:32:34.261300+0000","last_clean":"2023-10-06T04:32:34.261300+0000","last_became_active":"2023-10-06T04:32:24.266198+0000","last_became_peered":"2023-10-06T04:32:24.266198+0000","last_unstale":"2023-10-06T04:32:34.261300+0000","last_undegraded":"2023-10-06T04:32:34.261300+0000","last_fullsized":"2023-10-06T04:32:34.261300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315261+0000","last_change":"2023-10-06T04:32:24.265234+0000","last_active":"2023-10-06T04:32:35.315261+0000","last_peered":"2023-10-06T04:32:35.315261+0000","last_clean":"2023-10-06T04:32:35.315261+0000","last_became_active":"2023-10-06T04:32:24.264892+0000","last_became_peered":"2023-10-06T04:32:24.264892+0000","last_unstale":"2023-10-06T04:32:35.315261+0000","last_undegraded":"2023-10-06T04:32:35.315261+0000","last_fullsized":"2023-10-06T04:32:35.315261+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268527+0000","last_change":"2023-10-06T04:32:24.268696+0000","last_active":"2023-10-06T04:32:34.268527+0000","last_peered":"2023-10-06T04:32:34.268527+0000","last_clean":"2023-10-06T04:32:34.268527+0000","last_became_active":"2023-10-06T04:32:24.268430+0000","last_became_peered":"2023-10-06T04:32:24.268430+0000","last_unstale":"2023-10-06T04:32:34.268527+0000","last_undegraded":"2023-10-06T04:32:34.268527+0000","last_fullsized":"2023-10-06T04:32:34.268527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268528+0000","last_change":"2023-10-06T04:32:24.269272+0000","last_active":"2023-10-06T04:32:34.268528+0000","last_peered":"2023-10-06T04:32:34.268528+0000","last_clean":"2023-10-06T04:32:34.268528+0000","last_became_active":"2023-10-06T04:32:24.268793+0000","last_became_peered":"2023-10-06T04:32:24.268793+0000","last_unstale":"2023-10-06T04:32:34.268528+0000","last_undegraded":"2023-10-06T04:32:34.268528+0000","last_fullsized":"2023-10-06T04:32:34.268528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"34'10","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268890+0000","last_change":"2023-10-06T04:32:24.268089+0000","last_active":"2023-10-06T04:32:34.268890+0000","last_peered":"2023-10-06T04:32:34.268890+0000","last_clean":"2023-10-06T04:32:34.268890+0000","last_became_active":"2023-10-06T04:32:24.267842+0000","last_became_peered":"2023-10-06T04:32:24.267842+0000","last_unstale":"2023-10-06T04:32:34.268890+0000","last_undegraded":"2023-10-06T04:32:34.268890+0000","last_fullsized":"2023-10-06T04:32:34.268890+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:36.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"34'7","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263103+0000","last_change":"2023-10-06T04:32:24.263185+0000","last_active":"2023-10-06T04:32:34.263103+0000","last_peered":"2023-10-06T04:32:34.263103+0000","last_clean":"2023-10-06T04:32:34.263103+0000","last_became_active":"2023-10-06T04:32:24.262500+0000","last_became_peered":"2023-10-06T04:32:24.262500+0000","last_unstale":"2023-10-06T04:32:34.263103+0000","last_undegraded":"2023-10-06T04:32:34.263103+0000","last_fullsized":"2023-10-06T04:32:34.263103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:18.096304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268374+0000","last_change":"2023-10-06T04:32:24.266364+0000","last_active":"2023-10-06T04:32:34.268374+0000","last_peered":"2023-10-06T04:32:34.268374+0000","last_clean":"2023-10-06T04:32:34.268374+0000","last_became_active":"2023-10-06T04:32:24.266082+0000","last_became_peered":"2023-10-06T04:32:24.266082+0000","last_unstale":"2023-10-06T04:32:34.268374+0000","last_undegraded":"2023-10-06T04:32:34.268374+0000","last_fullsized":"2023-10-06T04:32:34.268374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"34'11","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315320+0000","last_change":"2023-10-06T04:32:24.265143+0000","last_active":"2023-10-06T04:32:35.315320+0000","last_peered":"2023-10-06T04:32:35.315320+0000","last_clean":"2023-10-06T04:32:35.315320+0000","last_became_active":"2023-10-06T04:32:24.264481+0000","last_became_peered":"2023-10-06T04:32:24.264481+0000","last_unstale":"2023-10-06T04:32:35.315320+0000","last_undegraded":"2023-10-06T04:32:35.315320+0000","last_fullsized":"2023-10-06T04:32:35.315320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:04.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260770+0000","last_change":"2023-10-06T04:32:24.262815+0000","last_active":"2023-10-06T04:32:34.260770+0000","last_peered":"2023-10-06T04:32:34.260770+0000","last_clean":"2023-10-06T04:32:34.260770+0000","last_became_active":"2023-10-06T04:32:24.262576+0000","last_became_peered":"2023-10-06T04:32:24.262576+0000","last_unstale":"2023-10-06T04:32:34.260770+0000","last_undegraded":"2023-10-06T04:32:34.260770+0000","last_fullsized":"2023-10-06T04:32:34.260770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"34'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269183+0000","last_change":"2023-10-06T04:32:24.269201+0000","last_active":"2023-10-06T04:32:34.269183+0000","last_peered":"2023-10-06T04:32:34.269183+0000","last_clean":"2023-10-06T04:32:34.269183+0000","last_became_active":"2023-10-06T04:32:24.268841+0000","last_became_peered":"2023-10-06T04:32:24.268841+0000","last_unstale":"2023-10-06T04:32:34.269183+0000","last_undegraded":"2023-10-06T04:32:34.269183+0000","last_fullsized":"2023-10-06T04:32:34.269183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"34'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262874+0000","last_change":"2023-10-06T04:32:24.264875+0000","last_active":"2023-10-06T04:32:34.262874+0000","last_peered":"2023-10-06T04:32:34.262874+0000","last_clean":"2023-10-06T04:32:34.262874+0000","last_became_active":"2023-10-06T04:32:24.264601+0000","last_became_peered":"2023-10-06T04:32:24.264601+0000","last_unstale":"2023-10-06T04:32:34.262874+0000","last_undegraded":"2023-10-06T04:32:34.262874+0000","last_fullsized":"2023-10-06T04:32:34.262874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:38.680988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"34'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269157+0000","last_change":"2023-10-06T04:32:24.268714+0000","last_active":"2023-10-06T04:32:34.269157+0000","last_peered":"2023-10-06T04:32:34.269157+0000","last_clean":"2023-10-06T04:32:34.269157+0000","last_became_active":"2023-10-06T04:32:24.268440+0000","last_became_peered":"2023-10-06T04:32:24.268440+0000","last_unstale":"2023-10-06T04:32:34.269157+0000","last_undegraded":"2023-10-06T04:32:34.269157+0000","last_fullsized":"2023-10-06T04:32:34.269157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:24.913639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268884+0000","last_change":"2023-10-06T04:32:24.266945+0000","last_active":"2023-10-06T04:32:34.268884+0000","last_peered":"2023-10-06T04:32:34.268884+0000","last_clean":"2023-10-06T04:32:34.268884+0000","last_became_active":"2023-10-06T04:32:24.266673+0000","last_became_peered":"2023-10-06T04:32:24.266673+0000","last_unstale":"2023-10-06T04:32:34.268884+0000","last_undegraded":"2023-10-06T04:32:34.268884+0000","last_fullsized":"2023-10-06T04:32:34.268884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"34'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268528+0000","last_change":"2023-10-06T04:32:24.270531+0000","last_active":"2023-10-06T04:32:34.268528+0000","last_peered":"2023-10-06T04:32:34.268528+0000","last_clean":"2023-10-06T04:32:34.268528+0000","last_became_active":"2023-10-06T04:32:24.270268+0000","last_became_peered":"2023-10-06T04:32:24.270268+0000","last_unstale":"2023-10-06T04:32:34.268528+0000","last_undegraded":"2023-10-06T04:32:34.268528+0000","last_fullsized":"2023-10-06T04:32:34.268528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:34.859497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"34'4","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315561+0000","last_change":"2023-10-06T04:32:24.265317+0000","last_active":"2023-10-06T04:32:35.315561+0000","last_peered":"2023-10-06T04:32:35.315561+0000","last_clean":"2023-10-06T04:32:35.315561+0000","last_became_active":"2023-10-06T04:32:24.264377+0000","last_became_peered":"2023-10-06T04:32:24.264377+0000","last_unstale":"2023-10-06T04:32:35.315561+0000","last_undegraded":"2023-10-06T04:32:35.315561+0000","last_fullsized":"2023-10-06T04:32:35.315561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"34'7","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268365+0000","last_change":"2023-10-06T04:32:24.264442+0000","last_active":"2023-10-06T04:32:34.268365+0000","last_peered":"2023-10-06T04:32:34.268365+0000","last_clean":"2023-10-06T04:32:34.268365+0000","last_became_active":"2023-10-06T04:32:24.264148+0000","last_became_peered":"2023-10-06T04:32:24.264148+0000","last_unstale":"2023-10-06T04:32:34.268365+0000","last_undegraded":"2023-10-06T04:32:34.268365+0000","last_fullsized":"2023-10-06T04:32:34.268365+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268651+0000","last_change":"2023-10-06T04:32:24.270800+0000","last_active":"2023-10-06T04:32:34.268651+0000","last_peered":"2023-10-06T04:32:34.268651+0000","last_clean":"2023-10-06T04:32:34.268651+0000","last_became_active":"2023-10-06T04:32:24.270560+0000","last_became_peered":"2023-10-06T04:32:24.270560+0000","last_unstale":"2023-10-06T04:32:34.268651+0000","last_undegraded":"2023-10-06T04:32:34.268651+0000","last_fullsized":"2023-10-06T04:32:34.268651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"34'12","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269073+0000","last_change":"2023-10-06T04:32:24.271609+0000","last_active":"2023-10-06T04:32:34.269073+0000","last_peered":"2023-10-06T04:32:34.269073+0000","last_clean":"2023-10-06T04:32:34.269073+0000","last_became_active":"2023-10-06T04:32:24.271354+0000","last_became_peered":"2023-10-06T04:32:24.271354+0000","last_unstale":"2023-10-06T04:32:34.269073+0000","last_undegraded":"2023-10-06T04:32:34.269073+0000","last_fullsized":"2023-10-06T04:32:34.269073+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270603+0000","last_change":"2023-10-06T04:32:24.263719+0000","last_active":"2023-10-06T04:32:34.270603+0000","last_peered":"2023-10-06T04:32:34.270603+0000","last_clean":"2023-10-06T04:32:34.270603+0000","last_became_active":"2023-10-06T04:32:24.263442+0000","last_became_peered":"2023-10-06T04:32:24.263442+0000","last_unstale":"2023-10-06T04:32:34.270603+0000","last_undegraded":"2023-10-06T04:32:34.270603+0000","last_fullsized":"2023-10-06T04:32:34.270603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:40.811547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268733+0000","last_change":"2023-10-06T04:32:24.270093+0000","last_active":"2023-10-06T04:32:34.268733+0000","last_peered":"2023-10-06T04:32:34.268733+0000","last_clean":"2023-10-06T04:32:34.268733+0000","last_became_active":"2023-10-06T04:32:24.269851+0000","last_became_peered":"2023-10-06T04:32:24.269851+0000","last_unstale":"2023-10-06T04:32:34.268733+0000","last_undegraded":"2023-10-06T04:32:34.268733+0000","last_fullsized":"2023-10-06T04:32:34.268733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'8","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269249+0000","last_change":"2023-10-06T04:32:23.441194+0000","last_active":"2023-10-06T04:32:34.269249+0000","last_peered":"2023-10-06T04:32:34.269249+0000","last_clean":"2023-10-06T04:32:34.269249+0000","last_became_active":"2023-10-06T04:32:23.440849+0000","last_became_peered":"2023-10-06T04:32:23.440849+0000","last_unstale":"2023-10-06T04:32:34.269249+0000","last_undegraded":"2023-10-06T04:32:34.269249+0000","last_fullsized":"2023-10-06T04:32:34.269249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:59.829363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268727+0000","last_change":"2023-10-06T04:32:24.267279+0000","last_active":"2023-10-06T04:32:34.268727+0000","last_peered":"2023-10-06T04:32:34.268727+0000","last_clean":"2023-10-06T04:32:34.268727+0000","last_became_active":"2023-10-06T04:32:24.267016+0000","last_became_peered":"2023-10-06T04:32:24.267016+0000","last_unstale":"2023-10-06T04:32:34.268727+0000","last_undegraded":"2023-10-06T04:32:34.268727+0000","last_fullsized":"2023-10-06T04:32:34.268727+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268798+0000","last_change":"2023-10-06T04:32:23.450031+0000","last_active":"2023-10-06T04:32:34.268798+0000","last_peered":"2023-10-06T04:32:34.268798+0000","last_clean":"2023-10-06T04:32:34.268798+0000","last_became_active":"2023-10-06T04:32:23.449717+0000","last_became_peered":"2023-10-06T04:32:23.449717+0000","last_unstale":"2023-10-06T04:32:34.268798+0000","last_undegraded":"2023-10-06T04:32:34.268798+0000","last_fullsized":"2023-10-06T04:32:34.268798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:19.596388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260810+0000","last_change":"2023-10-06T04:32:24.267302+0000","last_active":"2023-10-06T04:32:34.260810+0000","last_peered":"2023-10-06T04:32:34.260810+0000","last_clean":"2023-10-06T04:32:34.260810+0000","last_became_active":"2023-10-06T04:32:24.266976+0000","last_became_peered":"2023-10-06T04:32:24.266976+0000","last_unstale":"2023-10-06T04:32:34.260810+0000","last_undegraded":"2023-10-06T04:32:34.260810+0000","last_fullsized":"2023-10-06T04:32:34.260810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268890+0000","last_change":"2023-10-06T04:32:23.447029+0000","last_active":"2023-10-06T04:32:34.268890+0000","last_peered":"2023-10-06T04:32:34.268890+0000","last_clean":"2023-10-06T04:32:34.268890+0000","last_became_active":"2023-10-06T04:32:23.446659+0000","last_became_peered":"2023-10-06T04:32:23.446659+0000","last_unstale":"2023-10-06T04:32:34.268890+0000","last_undegraded":"2023-10-06T04:32:34.268890+0000","last_fullsized":"2023-10-06T04:32:34.268890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:13.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270340+0000","last_change":"2023-10-06T04:32:24.265927+0000","last_active":"2023-10-06T04:32:34.270340+0000","last_peered":"2023-10-06T04:32:34.270340+0000","last_clean":"2023-10-06T04:32:34.270340+0000","last_became_active":"2023-10-06T04:32:24.265668+0000","last_became_peered":"2023-10-06T04:32:24.265668+0000","last_unstale":"2023-10-06T04:32:34.270340+0000","last_undegraded":"2023-10-06T04:32:34.270340+0000","last_fullsized":"2023-10-06T04:32:34.270340+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260994+0000","last_change":"2023-10-06T04:32:23.437810+0000","last_active":"2023-10-06T04:32:34.260994+0000","last_peered":"2023-10-06T04:32:34.260994+0000","last_clean":"2023-10-06T04:32:34.260994+0000","last_became_active":"2023-10-06T04:32:23.437533+0000","last_became_peered":"2023-10-06T04:32:23.437533+0000","last_unstale":"2023-10-06T04:32:34.260994+0000","last_undegraded":"2023-10-06T04:32:34.260994+0000","last_fullsized":"2023-10-06T04:32:34.260994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:46.149228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268666+0000","last_change":"2023-10-06T04:32:24.261934+0000","last_active":"2023-10-06T04:32:34.268666+0000","last_peered":"2023-10-06T04:32:34.268666+0000","last_clean":"2023-10-06T04:32:34.268666+0000","last_became_active":"2023-10-06T04:32:24.261681+0000","last_became_peered":"2023-10-06T04:32:24.261681+0000","last_unstale":"2023-10-06T04:32:34.268666+0000","last_undegraded":"2023-10-06T04:32:34.268666+0000","last_fullsized":"2023-10-06T04:32:34.268666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315427+0000","last_change":"2023-10-06T04:32:23.443656+0000","last_active":"2023-10-06T04:32:35.315427+0000","last_peered":"2023-10-06T04:32:35.315427+0000","last_clean":"2023-10-06T04:32:35.315427+0000","last_became_active":"2023-10-06T04:32:23.443293+0000","last_became_peered":"2023-10-06T04:32:23.443293+0000","last_unstale":"2023-10-06T04:32:35.315427+0000","last_undegraded":"2023-10-06T04:32:35.315427+0000","last_fullsized":"2023-10-06T04:32:35.315427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315387+0000","last_change":"2023-10-06T04:32:24.264104+0000","last_active":"2023-10-06T04:32:35.315387+0000","last_peered":"2023-10-06T04:32:35.315387+0000","last_clean":"2023-10-06T04:32:35.315387+0000","last_became_active":"2023-10-06T04:32:24.263651+0000","last_became_peered":"2023-10-06T04:32:24.263651+0000","last_unstale":"2023-10-06T04:32:35.315387+0000","last_undegraded":"2023-10-06T04:32:35.315387+0000","last_fullsized":"2023-10-06T04:32:35.315387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:24.395312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269261+0000","last_change":"2023-10-06T04:32:23.451767+0000","last_active":"2023-10-06T04:32:34.269261+0000","last_peered":"2023-10-06T04:32:34.269261+0000","last_clean":"2023-10-06T04:32:34.269261+0000","last_became_active":"2023-10-06T04:32:23.451430+0000","last_became_peered":"2023-10-06T04:32:23.451430+0000","last_unstale":"2023-10-06T04:32:34.269261+0000","last_undegraded":"2023-10-06T04:32:34.269261+0000","last_fullsized":"2023-10-06T04:32:34.269261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261339+0000","last_change":"2023-10-06T04:32:24.267750+0000","last_active":"2023-10-06T04:32:34.261339+0000","last_peered":"2023-10-06T04:32:34.261339+0000","last_clean":"2023-10-06T04:32:34.261339+0000","last_became_active":"2023-10-06T04:32:24.265805+0000","last_became_peered":"2023-10-06T04:32:24.265805+0000","last_unstale":"2023-10-06T04:32:34.261339+0000","last_undegraded":"2023-10-06T04:32:34.261339+0000","last_fullsized":"2023-10-06T04:32:34.261339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268431+0000","last_change":"2023-10-06T04:32:23.449278+0000","last_active":"2023-10-06T04:32:34.268431+0000","last_peered":"2023-10-06T04:32:34.268431+0000","last_clean":"2023-10-06T04:32:34.268431+0000","last_became_active":"2023-10-06T04:32:23.449034+0000","last_became_peered":"2023-10-06T04:32:23.449034+0000","last_unstale":"2023-10-06T04:32:34.268431+0000","last_undegraded":"2023-10-06T04:32:34.268431+0000","last_fullsized":"2023-10-06T04:32:34.268431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269734+0000","last_change":"2023-10-06T04:32:24.265155+0000","last_active":"2023-10-06T04:32:34.269734+0000","last_peered":"2023-10-06T04:32:34.269734+0000","last_clean":"2023-10-06T04:32:34.269734+0000","last_became_active":"2023-10-06T04:32:24.264863+0000","last_became_peered":"2023-10-06T04:32:24.264863+0000","last_unstale":"2023-10-06T04:32:34.269734+0000","last_undegraded":"2023-10-06T04:32:34.269734+0000","last_fullsized":"2023-10-06T04:32:34.269734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260606+0000","last_change":"2023-10-06T04:32:23.440295+0000","last_active":"2023-10-06T04:32:34.260606+0000","last_peered":"2023-10-06T04:32:34.260606+0000","last_clean":"2023-10-06T04:32:34.260606+0000","last_became_active":"2023-10-06T04:32:23.440039+0000","last_became_peered":"2023-10-06T04:32:23.440039+0000","last_unstale":"2023-10-06T04:32:34.260606+0000","last_undegraded":"2023-10-06T04:32:34.260606+0000","last_fullsized":"2023-10-06T04:32:34.260606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"34'11","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263151+0000","last_change":"2023-10-06T04:32:24.271227+0000","last_active":"2023-10-06T04:32:34.263151+0000","last_peered":"2023-10-06T04:32:34.263151+0000","last_clean":"2023-10-06T04:32:34.263151+0000","last_became_active":"2023-10-06T04:32:24.270984+0000","last_became_peered":"2023-10-06T04:32:24.270984+0000","last_unstale":"2023-10-06T04:32:34.263151+0000","last_undegraded":"2023-10-06T04:32:34.263151+0000","last_fullsized":"2023-10-06T04:32:34.263151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270397+0000","last_change":"2023-10-06T04:32:23.447791+0000","last_active":"2023-10-06T04:32:34.270397+0000","last_peered":"2023-10-06T04:32:34.270397+0000","last_clean":"2023-10-06T04:32:34.270397+0000","last_became_active":"2023-10-06T04:32:23.447395+0000","last_became_peered":"2023-10-06T04:32:23.447395+0000","last_unstale":"2023-10-06T04:32:34.270397+0000","last_undegraded":"2023-10-06T04:32:34.270397+0000","last_fullsized":"2023-10-06T04:32:34.270397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:18.839316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"34'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269473+0000","last_change":"2023-10-06T04:32:24.269565+0000","last_active":"2023-10-06T04:32:34.269473+0000","last_peered":"2023-10-06T04:32:34.269473+0000","last_clean":"2023-10-06T04:32:34.269473+0000","last_became_active":"2023-10-06T04:32:24.269285+0000","last_became_peered":"2023-10-06T04:32:24.269285+0000","last_unstale":"2023-10-06T04:32:34.269473+0000","last_undegraded":"2023-10-06T04:32:34.269473+0000","last_fullsized":"2023-10-06T04:32:34.269473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315749+0000","last_change":"2023-10-06T04:32:23.444329+0000","last_active":"2023-10-06T04:32:35.315749+0000","last_peered":"2023-10-06T04:32:35.315749+0000","last_clean":"2023-10-06T04:32:35.315749+0000","last_became_active":"2023-10-06T04:32:23.444020+0000","last_became_peered":"2023-10-06T04:32:23.444020+0000","last_unstale":"2023-10-06T04:32:35.315749+0000","last_undegraded":"2023-10-06T04:32:35.315749+0000","last_fullsized":"2023-10-06T04:32:35.315749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"34'10","reported_seq":22,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268693+0000","last_change":"2023-10-06T04:32:24.264060+0000","last_active":"2023-10-06T04:32:34.268693+0000","last_peered":"2023-10-06T04:32:34.268693+0000","last_clean":"2023-10-06T04:32:34.268693+0000","last_became_active":"2023-10-06T04:32:24.263825+0000","last_became_peered":"2023-10-06T04:32:24.263825+0000","last_unstale":"2023-10-06T04:32:34.268693+0000","last_undegraded":"2023-10-06T04:32:34.268693+0000","last_fullsized":"2023-10-06T04:32:34.268693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'5","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315712+0000","last_change":"2023-10-06T04:32:23.447887+0000","last_active":"2023-10-06T04:32:35.315712+0000","last_peered":"2023-10-06T04:32:35.315712+0000","last_clean":"2023-10-06T04:32:35.315712+0000","last_became_active":"2023-10-06T04:32:23.447650+0000","last_became_peered":"2023-10-06T04:32:23.447650+0000","last_unstale":"2023-10-06T04:32:35.315712+0000","last_undegraded":"2023-10-06T04:32:35.315712+0000","last_fullsized":"2023-10-06T04:32:35.315712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:47.164678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260931+0000","last_change":"2023-10-06T04:32:24.270050+0000","last_active":"2023-10-06T04:32:34.260931+0000","last_peered":"2023-10-06T04:32:34.260931+0000","last_clean":"2023-10-06T04:32:34.260931+0000","last_became_active":"2023-10-06T04:32:24.269298+0000","last_became_peered":"2023-10-06T04:32:24.269298+0000","last_unstale":"2023-10-06T04:32:34.260931+0000","last_undegraded":"2023-10-06T04:32:34.260931+0000","last_fullsized":"2023-10-06T04:32:34.260931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'22","reported_seq":35,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269989+0000","last_change":"2023-10-06T04:32:23.447643+0000","last_active":"2023-10-06T04:32:34.269989+0000","last_peered":"2023-10-06T04:32:34.269989+0000","last_clean":"2023-10-06T04:32:34.269989+0000","last_became_active":"2023-10-06T04:32:23.447352+0000","last_became_peered":"2023-10-06T04:32:23.447352+0000","last_unstale":"2023-10-06T04:32:34.269989+0000","last_undegraded":"2023-10-06T04:32:34.269989+0000","last_fullsized":"2023-10-06T04:32:34.269989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268957+0000","last_change":"2023-10-06T04:32:24.267086+0000","last_active":"2023-10-06T04:32:34.268957+0000","last_peered":"2023-10-06T04:32:34.268957+0000","last_clean":"2023-10-06T04:32:34.268957+0000","last_became_active":"2023-10-06T04:32:24.266829+0000","last_became_peered":"2023-10-06T04:32:24.266829+0000","last_unstale":"2023-10-06T04:32:34.268957+0000","last_undegraded":"2023-10-06T04:32:34.268957+0000","last_fullsized":"2023-10-06T04:32:34.268957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'5","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269791+0000","last_change":"2023-10-06T04:32:23.443222+0000","last_active":"2023-10-06T04:32:34.269791+0000","last_peered":"2023-10-06T04:32:34.269791+0000","last_clean":"2023-10-06T04:32:34.269791+0000","last_became_active":"2023-10-06T04:32:23.442957+0000","last_became_peered":"2023-10-06T04:32:23.442957+0000","last_unstale":"2023-10-06T04:32:34.269791+0000","last_undegraded":"2023-10-06T04:32:34.269791+0000","last_fullsized":"2023-10-06T04:32:34.269791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269309+0000","last_change":"2023-10-06T04:32:24.268660+0000","last_active":"2023-10-06T04:32:34.269309+0000","last_peered":"2023-10-06T04:32:34.269309+0000","last_clean":"2023-10-06T04:32:34.269309+0000","last_became_active":"2023-10-06T04:32:24.268397+0000","last_became_peered":"2023-10-06T04:32:24.268397+0000","last_unstale":"2023-10-06T04:32:34.269309+0000","last_undegraded":"2023-10-06T04:32:34.269309+0000","last_fullsized":"2023-10-06T04:32:34.269309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268494+0000","last_change":"2023-10-06T04:32:23.445203+0000","last_active":"2023-10-06T04:32:34.268494+0000","last_peered":"2023-10-06T04:32:34.268494+0000","last_clean":"2023-10-06T04:32:34.268494+0000","last_became_active":"2023-10-06T04:32:23.444688+0000","last_became_peered":"2023-10-06T04:32:23.444688+0000","last_unstale":"2023-10-06T04:32:34.268494+0000","last_undegraded":"2023-10-06T04:32:34.268494+0000","last_fullsized":"2023-10-06T04:32:34.268494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269146+0000","last_change":"2023-10-06T04:32:24.271729+0000","last_active":"2023-10-06T04:32:34.269146+0000","last_peered":"2023-10-06T04:32:34.269146+0000","last_clean":"2023-10-06T04:32:34.269146+0000","last_became_active":"2023-10-06T04:32:24.271453+0000","last_became_peered":"2023-10-06T04:32:24.271453+0000","last_unstale":"2023-10-06T04:32:34.269146+0000","last_undegraded":"2023-10-06T04:32:34.269146+0000","last_fullsized":"2023-10-06T04:32:34.269146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268204+0000","last_change":"2023-10-06T04:32:23.440530+0000","last_active":"2023-10-06T04:32:34.268204+0000","last_peered":"2023-10-06T04:32:34.268204+0000","last_clean":"2023-10-06T04:32:34.268204+0000","last_became_active":"2023-10-06T04:32:23.440226+0000","last_became_peered":"2023-10-06T04:32:23.440226+0000","last_unstale":"2023-10-06T04:32:34.268204+0000","last_undegraded":"2023-10-06T04:32:34.268204+0000","last_fullsized":"2023-10-06T04:32:34.268204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269621+0000","last_change":"2023-10-06T04:32:24.268363+0000","last_active":"2023-10-06T04:32:34.269621+0000","last_peered":"2023-10-06T04:32:34.269621+0000","last_clean":"2023-10-06T04:32:34.269621+0000","last_became_active":"2023-10-06T04:32:24.268116+0000","last_became_peered":"2023-10-06T04:32:24.268116+0000","last_unstale":"2023-10-06T04:32:34.269621+0000","last_undegraded":"2023-10-06T04:32:34.269621+0000","last_fullsized":"2023-10-06T04:32:34.269621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'15","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268764+0000","last_change":"2023-10-06T04:32:23.452500+0000","last_active":"2023-10-06T04:32:34.268764+0000","last_peered":"2023-10-06T04:32:34.268764+0000","last_clean":"2023-10-06T04:32:34.268764+0000","last_became_active":"2023-10-06T04:32:23.452236+0000","last_became_peered":"2023-10-06T04:32:23.452236+0000","last_unstale":"2023-10-06T04:32:34.268764+0000","last_undegraded":"2023-10-06T04:32:34.268764+0000","last_fullsized":"2023-10-06T04:32:34.268764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269934+0000","last_change":"2023-10-06T04:32:24.265984+0000","last_active":"2023-10-06T04:32:34.269934+0000","last_peered":"2023-10-06T04:32:34.269934+0000","last_clean":"2023-10-06T04:32:34.269934+0000","last_became_active":"2023-10-06T04:32:24.265707+0000","last_became_peered":"2023-10-06T04:32:24.265707+0000","last_unstale":"2023-10-06T04:32:34.269934+0000","last_undegraded":"2023-10-06T04:32:34.269934+0000","last_fullsized":"2023-10-06T04:32:34.269934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315804+0000","last_change":"2023-10-06T04:32:23.447020+0000","last_active":"2023-10-06T04:32:35.315804+0000","last_peered":"2023-10-06T04:32:35.315804+0000","last_clean":"2023-10-06T04:32:35.315804+0000","last_became_active":"2023-10-06T04:32:23.446786+0000","last_became_peered":"2023-10-06T04:32:23.446786+0000","last_unstale":"2023-10-06T04:32:35.315804+0000","last_undegraded":"2023-10-06T04:32:35.315804+0000","last_fullsized":"2023-10-06T04:32:35.315804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.805674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315600+0000","last_change":"2023-10-06T04:32:24.265722+0000","last_active":"2023-10-06T04:32:35.315600+0000","last_peered":"2023-10-06T04:32:35.315600+0000","last_clean":"2023-10-06T04:32:35.315600+0000","last_became_active":"2023-10-06T04:32:24.265435+0000","last_became_peered":"2023-10-06T04:32:24.265435+0000","last_unstale":"2023-10-06T04:32:35.315600+0000","last_undegraded":"2023-10-06T04:32:35.315600+0000","last_fullsized":"2023-10-06T04:32:35.315600+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.261738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260867+0000","last_change":"2023-10-06T04:32:23.442348+0000","last_active":"2023-10-06T04:32:34.260867+0000","last_peered":"2023-10-06T04:32:34.260867+0000","last_clean":"2023-10-06T04:32:34.260867+0000","last_became_active":"2023-10-06T04:32:23.441833+0000","last_became_peered":"2023-10-06T04:32:23.441833+0000","last_unstale":"2023-10-06T04:32:34.260867+0000","last_undegraded":"2023-10-06T04:32:34.260867+0000","last_fullsized":"2023-10-06T04:32:34.260867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.916159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260744+0000","last_change":"2023-10-06T04:32:24.254622+0000","last_active":"2023-10-06T04:32:34.260744+0000","last_peered":"2023-10-06T04:32:34.260744+0000","last_clean":"2023-10-06T04:32:34.260744+0000","last_became_active":"2023-10-06T04:32:24.254294+0000","last_became_peered":"2023-10-06T04:32:24.254294+0000","last_unstale":"2023-10-06T04:32:34.260744+0000","last_undegraded":"2023-10-06T04:32:34.260744+0000","last_fullsized":"2023-10-06T04:32:34.260744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263152+0000","last_change":"2023-10-06T04:32:23.447777+0000","last_active":"2023-10-06T04:32:34.263152+0000","last_peered":"2023-10-06T04:32:34.263152+0000","last_clean":"2023-10-06T04:32:34.263152+0000","last_became_active":"2023-10-06T04:32:23.447526+0000","last_became_peered":"2023-10-06T04:32:23.447526+0000","last_unstale":"2023-10-06T04:32:34.263152+0000","last_undegraded":"2023-10-06T04:32:34.263152+0000","last_fullsized":"2023-10-06T04:32:34.263152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:59.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"34'9","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263255+0000","last_change":"2023-10-06T04:32:24.270632+0000","last_active":"2023-10-06T04:32:34.263255+0000","last_peered":"2023-10-06T04:32:34.263255+0000","last_clean":"2023-10-06T04:32:34.263255+0000","last_became_active":"2023-10-06T04:32:24.270263+0000","last_became_peered":"2023-10-06T04:32:24.270263+0000","last_unstale":"2023-10-06T04:32:34.263255+0000","last_undegraded":"2023-10-06T04:32:34.263255+0000","last_fullsized":"2023-10-06T04:32:34.263255+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:19.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268936+0000","last_change":"2023-10-06T04:32:23.446504+0000","last_active":"2023-10-06T04:32:34.268936+0000","last_peered":"2023-10-06T04:32:34.268936+0000","last_clean":"2023-10-06T04:32:34.268936+0000","last_became_active":"2023-10-06T04:32:23.446221+0000","last_became_peered":"2023-10-06T04:32:23.446221+0000","last_unstale":"2023-10-06T04:32:34.268936+0000","last_undegraded":"2023-10-06T04:32:34.268936+0000","last_fullsized":"2023-10-06T04:32:34.268936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268647+0000","last_change":"2023-10-06T04:32:24.271430+0000","last_active":"2023-10-06T04:32:34.268647+0000","last_peered":"2023-10-06T04:32:34.268647+0000","last_clean":"2023-10-06T04:32:34.268647+0000","last_became_active":"2023-10-06T04:32:24.271180+0000","last_became_peered":"2023-10-06T04:32:24.271180+0000","last_unstale":"2023-10-06T04:32:34.268647+0000","last_undegraded":"2023-10-06T04:32:34.268647+0000","last_fullsized":"2023-10-06T04:32:34.268647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'4","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263236+0000","last_change":"2023-10-06T04:32:23.445911+0000","last_active":"2023-10-06T04:32:34.263236+0000","last_peered":"2023-10-06T04:32:34.263236+0000","last_clean":"2023-10-06T04:32:34.263236+0000","last_became_active":"2023-10-06T04:32:23.445582+0000","last_became_peered":"2023-10-06T04:32:23.445582+0000","last_unstale":"2023-10-06T04:32:34.263236+0000","last_undegraded":"2023-10-06T04:32:34.263236+0000","last_fullsized":"2023-10-06T04:32:34.263236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:35.404671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"34'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269245+0000","last_change":"2023-10-06T04:32:24.259250+0000","last_active":"2023-10-06T04:32:34.269245+0000","last_peered":"2023-10-06T04:32:34.269245+0000","last_clean":"2023-10-06T04:32:34.269245+0000","last_became_active":"2023-10-06T04:32:24.258962+0000","last_became_peered":"2023-10-06T04:32:24.258962+0000","last_unstale":"2023-10-06T04:32:34.269245+0000","last_undegraded":"2023-10-06T04:32:34.269245+0000","last_fullsized":"2023-10-06T04:32:34.269245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262972+0000","last_change":"2023-10-06T04:32:23.445529+0000","last_active":"2023-10-06T04:32:34.262972+0000","last_peered":"2023-10-06T04:32:34.262972+0000","last_clean":"2023-10-06T04:32:34.262972+0000","last_became_active":"2023-10-06T04:32:23.445245+0000","last_became_peered":"2023-10-06T04:32:23.445245+0000","last_unstale":"2023-10-06T04:32:34.262972+0000","last_undegraded":"2023-10-06T04:32:34.262972+0000","last_fullsized":"2023-10-06T04:32:34.262972+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315277+0000","last_change":"2023-10-06T04:32:24.270658+0000","last_active":"2023-10-06T04:32:35.315277+0000","last_peered":"2023-10-06T04:32:35.315277+0000","last_clean":"2023-10-06T04:32:35.315277+0000","last_became_active":"2023-10-06T04:32:24.270394+0000","last_became_peered":"2023-10-06T04:32:24.270394+0000","last_unstale":"2023-10-06T04:32:35.315277+0000","last_undegraded":"2023-10-06T04:32:35.315277+0000","last_fullsized":"2023-10-06T04:32:35.315277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269283+0000","last_change":"2023-10-06T04:32:23.452178+0000","last_active":"2023-10-06T04:32:34.269283+0000","last_peered":"2023-10-06T04:32:34.269283+0000","last_clean":"2023-10-06T04:32:34.269283+0000","last_became_active":"2023-10-06T04:32:23.451849+0000","last_became_peered":"2023-10-06T04:32:23.451849+0000","last_unstale":"2023-10-06T04:32:34.269283+0000","last_undegraded":"2023-10-06T04:32:34.269283+0000","last_fullsized":"2023-10-06T04:32:34.269283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.026875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269679+0000","last_change":"2023-10-06T04:32:24.265320+0000","last_active":"2023-10-06T04:32:34.269679+0000","last_peered":"2023-10-06T04:32:34.269679+0000","last_clean":"2023-10-06T04:32:34.269679+0000","last_became_active":"2023-10-06T04:32:24.264894+0000","last_became_peered":"2023-10-06T04:32:24.264894+0000","last_unstale":"2023-10-06T04:32:34.269679+0000","last_undegraded":"2023-10-06T04:32:34.269679+0000","last_fullsized":"2023-10-06T04:32:34.269679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262902+0000","last_change":"2023-10-06T04:32:23.444136+0000","last_active":"2023-10-06T04:32:34.262902+0000","last_peered":"2023-10-06T04:32:34.262902+0000","last_clean":"2023-10-06T04:32:34.262902+0000","last_became_active":"2023-10-06T04:32:23.443839+0000","last_became_peered":"2023-10-06T04:32:23.443839+0000","last_unstale":"2023-10-06T04:32:34.262902+0000","last_undegraded":"2023-10-06T04:32:34.262902+0000","last_fullsized":"2023-10-06T04:32:34.262902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:10.814479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268848+0000","last_change":"2023-10-06T04:32:24.269672+0000","last_active":"2023-10-06T04:32:34.268848+0000","last_peered":"2023-10-06T04:32:34.268848+0000","last_clean":"2023-10-06T04:32:34.268848+0000","last_became_active":"2023-10-06T04:32:24.269407+0000","last_became_peered":"2023-10-06T04:32:24.269407+0000","last_unstale":"2023-10-06T04:32:34.268848+0000","last_undegraded":"2023-10-06T04:32:34.268848+0000","last_fullsized":"2023-10-06T04:32:34.268848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269617+0000","last_change":"2023-10-06T04:32:23.446685+0000","last_active":"2023-10-06T04:32:34.269617+0000","last_peered":"2023-10-06T04:32:34.269617+0000","last_clean":"2023-10-06T04:32:34.269617+0000","last_became_active":"2023-10-06T04:32:23.446432+0000","last_became_peered":"2023-10-06T04:32:23.446432+0000","last_unstale":"2023-10-06T04:32:34.269617+0000","last_undegraded":"2023-10-06T04:32:34.269617+0000","last_fullsized":"2023-10-06T04:32:34.269617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268778+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:32:34.268778+0000","last_peered":"2023-10-06T04:32:34.268778+0000","last_clean":"2023-10-06T04:32:34.268778+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:32:34.268778+0000","last_undegraded":"2023-10-06T04:32:34.268778+0000","last_fullsized":"2023-10-06T04:32:34.268778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270090+0000","last_change":"2023-10-06T04:32:24.266084+0000","last_active":"2023-10-06T04:32:34.270090+0000","last_peered":"2023-10-06T04:32:34.270090+0000","last_clean":"2023-10-06T04:32:34.270090+0000","last_became_active":"2023-10-06T04:32:24.265847+0000","last_became_peered":"2023-10-06T04:32:24.265847+0000","last_unstale":"2023-10-06T04:32:34.270090+0000","last_undegraded":"2023-10-06T04:32:34.270090+0000","last_fullsized":"2023-10-06T04:32:34.270090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270247+0000","last_change":"2023-10-06T04:32:23.451106+0000","last_active":"2023-10-06T04:32:34.270247+0000","last_peered":"2023-10-06T04:32:34.270247+0000","last_clean":"2023-10-06T04:32:34.270247+0000","last_became_active":"2023-10-06T04:32:23.450838+0000","last_became_peered":"2023-10-06T04:32:23.450838+0000","last_unstale":"2023-10-06T04:32:34.270247+0000","last_undegraded":"2023-10-06T04:32:34.270247+0000","last_fullsized":"2023-10-06T04:32:34.270247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315391+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:32:35.315391+0000","last_peered":"2023-10-06T04:32:35.315391+0000","last_clean":"2023-10-06T04:32:35.315391+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:32:35.315391+0000","last_undegraded":"2023-10-06T04:32:35.315391+0000","last_fullsized":"2023-10-06T04:32:35.315391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269578+0000","last_change":"2023-10-06T04:32:24.272289+0000","last_active":"2023-10-06T04:32:34.269578+0000","last_peered":"2023-10-06T04:32:34.269578+0000","last_clean":"2023-10-06T04:32:34.269578+0000","last_became_active":"2023-10-06T04:32:24.271996+0000","last_became_peered":"2023-10-06T04:32:24.271996+0000","last_unstale":"2023-10-06T04:32:34.269578+0000","last_undegraded":"2023-10-06T04:32:34.269578+0000","last_fullsized":"2023-10-06T04:32:34.269578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'3","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263158+0000","last_change":"2023-10-06T04:32:23.444286+0000","last_active":"2023-10-06T04:32:34.263158+0000","last_peered":"2023-10-06T04:32:34.263158+0000","last_clean":"2023-10-06T04:32:34.263158+0000","last_became_active":"2023-10-06T04:32:23.443982+0000","last_became_peered":"2023-10-06T04:32:23.443982+0000","last_unstale":"2023-10-06T04:32:34.263158+0000","last_undegraded":"2023-10-06T04:32:34.263158+0000","last_fullsized":"2023-10-06T04:32:34.263158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:20.394340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262988+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:32:34.262988+0000","last_peered":"2023-10-06T04:32:34.262988+0000","last_clean":"2023-10-06T04:32:34.262988+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:32:34.262988+0000","last_undegraded":"2023-10-06T04:32:34.262988+0000","last_fullsized":"2023-10-06T04:32:34.262988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"34'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315767+0000","last_change":"2023-10-06T04:32:24.265180+0000","last_active":"2023-10-06T04:32:35.315767+0000","last_peered":"2023-10-06T04:32:35.315767+0000","last_clean":"2023-10-06T04:32:35.315767+0000","last_became_active":"2023-10-06T04:32:24.264769+0000","last_became_peered":"2023-10-06T04:32:24.264769+0000","last_unstale":"2023-10-06T04:32:35.315767+0000","last_undegraded":"2023-10-06T04:32:35.315767+0000","last_fullsized":"2023-10-06T04:32:35.315767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268997+0000","last_change":"2023-10-06T04:32:23.452281+0000","last_active":"2023-10-06T04:32:34.268997+0000","last_peered":"2023-10-06T04:32:34.268997+0000","last_clean":"2023-10-06T04:32:34.268997+0000","last_became_active":"2023-10-06T04:32:23.452023+0000","last_became_peered":"2023-10-06T04:32:23.452023+0000","last_unstale":"2023-10-06T04:32:34.268997+0000","last_undegraded":"2023-10-06T04:32:34.268997+0000","last_fullsized":"2023-10-06T04:32:34.268997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315453+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:32:35.315453+0000","last_peered":"2023-10-06T04:32:35.315453+0000","last_clean":"2023-10-06T04:32:35.315453+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:32:35.315453+0000","last_undegraded":"2023-10-06T04:32:35.315453+0000","last_fullsized":"2023-10-06T04:32:35.315453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269424+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:32:34.269424+0000","last_peered":"2023-10-06T04:32:34.269424+0000","last_clean":"2023-10-06T04:32:34.269424+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:32:34.269424+0000","last_undegraded":"2023-10-06T04:32:34.269424+0000","last_fullsized":"2023-10-06T04:32:34.269424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268400+0000","last_change":"2023-10-06T04:32:24.267047+0000","last_active":"2023-10-06T04:32:34.268400+0000","last_peered":"2023-10-06T04:32:34.268400+0000","last_clean":"2023-10-06T04:32:34.268400+0000","last_became_active":"2023-10-06T04:32:24.266777+0000","last_became_peered":"2023-10-06T04:32:24.266777+0000","last_unstale":"2023-10-06T04:32:34.268400+0000","last_undegraded":"2023-10-06T04:32:34.268400+0000","last_fullsized":"2023-10-06T04:32:34.268400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315859+0000","last_change":"2023-10-06T04:32:23.448480+0000","last_active":"2023-10-06T04:32:35.315859+0000","last_peered":"2023-10-06T04:32:35.315859+0000","last_clean":"2023-10-06T04:32:35.315859+0000","last_became_active":"2023-10-06T04:32:23.448242+0000","last_became_peered":"2023-10-06T04:32:23.448242+0000","last_unstale":"2023-10-06T04:32:35.315859+0000","last_undegraded":"2023-10-06T04:32:35.315859+0000","last_fullsized":"2023-10-06T04:32:35.315859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269418+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:32:34.269418+0000","last_peered":"2023-10-06T04:32:34.269418+0000","last_clean":"2023-10-06T04:32:34.269418+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:32:34.269418+0000","last_undegraded":"2023-10-06T04:32:34.269418+0000","last_fullsized":"2023-10-06T04:32:34.269418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268375+0000","last_change":"2023-10-06T04:32:24.266050+0000","last_active":"2023-10-06T04:32:34.268375+0000","last_peered":"2023-10-06T04:32:34.268375+0000","last_clean":"2023-10-06T04:32:34.268375+0000","last_became_active":"2023-10-06T04:32:24.265769+0000","last_became_peered":"2023-10-06T04:32:24.265769+0000","last_unstale":"2023-10-06T04:32:34.268375+0000","last_undegraded":"2023-10-06T04:32:34.268375+0000","last_fullsized":"2023-10-06T04:32:34.268375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:01.427293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268726+0000","last_change":"2023-10-06T04:32:23.443114+0000","last_active":"2023-10-06T04:32:34.268726+0000","last_peered":"2023-10-06T04:32:34.268726+0000","last_clean":"2023-10-06T04:32:34.268726+0000","last_became_active":"2023-10-06T04:32:23.442844+0000","last_became_peered":"2023-10-06T04:32:23.442844+0000","last_unstale":"2023-10-06T04:32:34.268726+0000","last_undegraded":"2023-10-06T04:32:34.268726+0000","last_fullsized":"2023-10-06T04:32:34.268726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269667+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:32:34.269667+0000","last_peered":"2023-10-06T04:32:34.269667+0000","last_clean":"2023-10-06T04:32:34.269667+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:32:34.269667+0000","last_undegraded":"2023-10-06T04:32:34.269667+0000","last_fullsized":"2023-10-06T04:32:34.269667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261395+0000","last_change":"2023-10-06T04:32:24.265505+0000","last_active":"2023-10-06T04:32:34.261395+0000","last_peered":"2023-10-06T04:32:34.261395+0000","last_clean":"2023-10-06T04:32:34.261395+0000","last_became_active":"2023-10-06T04:32:24.265228+0000","last_became_peered":"2023-10-06T04:32:24.265228+0000","last_unstale":"2023-10-06T04:32:34.261395+0000","last_undegraded":"2023-10-06T04:32:34.261395+0000","last_fullsized":"2023-10-06T04:32:34.261395+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260966+0000","last_change":"2023-10-06T04:32:23.438229+0000","last_active":"2023-10-06T04:32:34.260966+0000","last_peered":"2023-10-06T04:32:34.260966+0000","last_clean":"2023-10-06T04:32:34.260966+0000","last_became_active":"2023-10-06T04:32:23.437954+0000","last_became_peered":"2023-10-06T04:32:23.437954+0000","last_unstale":"2023-10-06T04:32:34.260966+0000","last_undegraded":"2023-10-06T04:32:34.260966+0000","last_fullsized":"2023-10-06T04:32:34.260966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:22.972008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268724+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:32:34.268724+0000","last_peered":"2023-10-06T04:32:34.268724+0000","last_clean":"2023-10-06T04:32:34.268724+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:32:34.268724+0000","last_undegraded":"2023-10-06T04:32:34.268724+0000","last_fullsized":"2023-10-06T04:32:34.268724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269083+0000","last_change":"2023-10-06T04:32:24.272223+0000","last_active":"2023-10-06T04:32:34.269083+0000","last_peered":"2023-10-06T04:32:34.269083+0000","last_clean":"2023-10-06T04:32:34.269083+0000","last_became_active":"2023-10-06T04:32:24.271905+0000","last_became_peered":"2023-10-06T04:32:24.271905+0000","last_unstale":"2023-10-06T04:32:34.269083+0000","last_undegraded":"2023-10-06T04:32:34.269083+0000","last_fullsized":"2023-10-06T04:32:34.269083+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:17.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269275+0000","last_change":"2023-10-06T04:32:23.444321+0000","last_active":"2023-10-06T04:32:34.269275+0000","last_peered":"2023-10-06T04:32:34.269275+0000","last_clean":"2023-10-06T04:32:34.269275+0000","last_became_active":"2023-10-06T04:32:23.444073+0000","last_became_peered":"2023-10-06T04:32:23.444073+0000","last_unstale":"2023-10-06T04:32:34.269275+0000","last_undegraded":"2023-10-06T04:32:34.269275+0000","last_fullsized":"2023-10-06T04:32:34.269275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:37.675115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269325+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:32:34.269325+0000","last_peered":"2023-10-06T04:32:34.269325+0000","last_clean":"2023-10-06T04:32:34.269325+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:32:34.269325+0000","last_undegraded":"2023-10-06T04:32:34.269325+0000","last_fullsized":"2023-10-06T04:32:34.269325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300340,"kb_used_data":3052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071500,"statfs":{"total":96636764160,"available":96329216000,"internally_reserved":0,"allocated":3125248,"data_stored":2869386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":188202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.496},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.81100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297700,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074140,"statfs":{"total":96636764160,"available":96331919360,"internally_reserved":0,"allocated":421888,"data_stored":175530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.88600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073536,"statfs":{"total":96636764160,"available":96331300864,"internally_reserved":0,"allocated":1040384,"data_stored":778570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.26100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.27200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297712,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074128,"statfs":{"total":96636764160,"available":96331907072,"internally_reserved":0,"allocated":434176,"data_stored":181866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.23100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.371}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301848,"kb_used_data":4532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069992,"statfs":{"total":96636764160,"available":96327671808,"internally_reserved":0,"allocated":4640768,"data_stored":4382506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.248},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.29999999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":188202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.495}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36299999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303844,"kb_used_data":6556,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067996,"statfs":{"total":96636764160,"available":96325627904,"internally_reserved":0,"allocated":6713344,"data_stored":6466986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.626}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:32:41.671 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:32:41.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.817+0000 7f92b86a5700 1 Processor -- start 2023-10-06T04:32:41.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.817+0000 7f92b86a5700 1 -- start start 2023-10-06T04:32:41.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.818+0000 7f92b86a5700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f92b00c9570 0x7f92b00a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.818+0000 7f92b86a5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 0x7f92b00a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.818+0000 7f92b86a5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b006a750 con 0x7f92b0067660 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.818+0000 7f92b86a5700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b006aa30 con 0x7f92b00c9570 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.818+0000 7f92b86a5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b006ad10 con 0x7f92b00ca5b0 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.818+0000 7f92b6441700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f92b0067660 0x7f92b0067a30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45040/0 (socket says 172.21.15.123:45040) 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.818+0000 7f92b6441700 1 -- 172.21.15.123:0/2847706864 learned_addr learned my addr 172.21.15.123:0/2847706864 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 0x7f92b00a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b5c40700 1 --2- 172.21.15.123:0/2847706864 >> v2:172.21.15.177:3300/0 conn(0x7f92b00c9570 0x7f92b00a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073460959 0 0) 0x7f92b006a750 con 0x7f92b0067660 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f929c003570 con 0x7f92b0067660 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 967985755 0 0) 0x7f929c003570 con 0x7f92b0067660 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 msgr2=0x7f92b00a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 0x7f92b00a9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 >> v2:172.21.15.177:3300/0 conn(0x7f92b00c9570 msgr2=0x7f92b00a9940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 --2- 172.21.15.123:0/2847706864 >> v2:172.21.15.177:3300/0 conn(0x7f92b00c9570 0x7f92b00a9940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b006aff0 con 0x7f92b0067660 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f92a0001470 con 0x7f92b0067660 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 >> v1:172.21.15.123:6789/0 conn(0x7f92b0067660 legacy=0x7f92b0067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c003a60 0x7f929c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 0x7f929c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f929c004d50 0x7f929c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b006aa30 con 0x7f929c004d50 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b006ad10 con 0x7f929c003a60 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f929c00ba60 con 0x7f92b00ca5b0 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f929c004d50 0x7f929c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b5c40700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 0x7f929c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6441700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c003a60 0x7f929c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 -- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 msgr2=0x7f929c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00ca5b0 0x7f929c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 -- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c003a60 msgr2=0x7f929c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c003a60 0x7f929c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 -- 172.21.15.123:0/2847706864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929c003570 con 0x7f929c004d50 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.819+0000 7f92b6c42700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f929c004d50 0x7f929c0051c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e210 tx=0x7f92a800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7266ce822d65f52b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800ea60 con 0x7f929c004d50 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a800f040 con 0x7f929c004d50 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b543f700 1 -- 172.21.15.123:0/2847706864 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800ee10 con 0x7f929c004d50 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b86a5700 1 -- 172.21.15.123:0/2847706864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f929c004d50 msgr2=0x7f929c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b86a5700 1 --2- 172.21.15.123:0/2847706864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f929c004d50 0x7f929c0051c0 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f92a800e210 tx=0x7f92a800b040 comp rx=0 tx=0).stop 2023-10-06T04:32:41.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b6c42700 1 -- 172.21.15.123:0/2847706864 reap_dead start 2023-10-06T04:32:41.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b86a5700 1 -- 172.21.15.123:0/2847706864 shutdown_connections 2023-10-06T04:32:41.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b86a5700 1 -- 172.21.15.123:0/2847706864 >> 172.21.15.123:0/2847706864 conn(0x7f92b00cda80 msgr2=0x7f92b00c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:41.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b86a5700 1 -- 172.21.15.123:0/2847706864 shutdown_connections 2023-10-06T04:32:41.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.820+0000 7f92b86a5700 1 -- 172.21.15.123:0/2847706864 wait complete. 2023-10-06T04:32:41.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.821+0000 7f92b86a5700 1 Processor -- start 2023-10-06T04:32:41.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.821+0000 7f92b86a5700 1 -- start start 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b86a5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c004d50 0x7f92b00ad260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b86a5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00af1d0 0x7f92b0151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b86a5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b00c2a30 0x7f92b00c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b86a5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b00c0a50 con 0x7f92b00c2a30 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b86a5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b00c33c0 con 0x7f929c004d50 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b86a5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b00ad960 con 0x7f92b00af1d0 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b00c2a30 0x7f92b00c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b00c2a30 0x7f92b00c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33608/0 (socket says 172.21.15.123:33608) 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 -- 172.21.15.123:0/3006568413 learned_addr learned my addr 172.21.15.123:0/3006568413 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:41.831 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b5c40700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00af1d0 0x7f92b0151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6441700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c004d50 0x7f92b00ad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 -- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00af1d0 msgr2=0x7f92b0151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00af1d0 0x7f92b0151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 -- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c004d50 msgr2=0x7f92b00ad260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c004d50 0x7f92b00ad260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.822+0000 7f92b6c42700 1 -- 172.21.15.123:0/3006568413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b00adc40 con 0x7f92b00c2a30 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.823+0000 7f92b6c42700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b00c2a30 0x7f92b00c2e80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f92a800afd0 tx=0x7f92a801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.823+0000 7f92a67fc700 1 -- 172.21.15.123:0/3006568413 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a800eee0 con 0x7f92b00c2a30 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.823+0000 7f92a67fc700 1 -- 172.21.15.123:0/3006568413 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a8022070 con 0x7f92b00c2a30 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.823+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b00adf20 con 0x7f92b00c2a30 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.823+0000 7f92a67fc700 1 -- 172.21.15.123:0/3006568413 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a801e870 con 0x7f92b00c2a30 2023-10-06T04:32:41.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.823+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b00af640 con 0x7f92b00c2a30 2023-10-06T04:32:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.824+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b0158d00 con 0x7f92b00c2a30 2023-10-06T04:32:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.824+0000 7f92a67fc700 1 -- 172.21.15.123:0/3006568413 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92a8003040 con 0x7f92b00c2a30 2023-10-06T04:32:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.825+0000 7f92a67fc700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9294066f30 0x7f92940693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.825+0000 7f92a67fc700 1 -- 172.21.15.123:0/3006568413 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f92a8024070 con 0x7f92b00c2a30 2023-10-06T04:32:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.825+0000 7f92b6441700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9294066f30 0x7f92940693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.825+0000 7f92b6441700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9294066f30 0x7f92940693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92a0005d40 tx=0x7f92a0007040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:41.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:41.829+0000 7f92a67fc700 1 -- 172.21.15.123:0/3006568413 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92a8053300 con 0x7f92b00c2a30 2023-10-06T04:32:42.010 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.007+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f92b00cde80 con 0x7f9294066f30 2023-10-06T04:32:42.019 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.017+0000 7f92a67fc700 1 -- 172.21.15.123:0/3006568413 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+386486 (secure 0 0 0) 0x7f92b00cde80 con 0x7f9294066f30 2023-10-06T04:32:42.025 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9294066f30 msgr2=0x7f92940693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9294066f30 0x7f92940693e0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f92a0005d40 tx=0x7f92a0007040 comp rx=0 tx=0).stop 2023-10-06T04:32:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b00c2a30 msgr2=0x7f92b00c2e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b00c2a30 0x7f92b00c2e80 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7f92a800afd0 tx=0x7f92a801a000 comp rx=0 tx=0).stop 2023-10-06T04:32:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 shutdown_connections 2023-10-06T04:32:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9294066f30 0x7f92940693e0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92b00c2a30 0x7f92b00c2e80 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f929c004d50 0x7f92b00ad260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 --2- 172.21.15.123:0/3006568413 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92b00af1d0 0x7f92b0151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 >> 172.21.15.123:0/3006568413 conn(0x7f92b00cda80 msgr2=0x7f92b00a9960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:42.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 shutdown_connections 2023-10-06T04:32:42.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.021+0000 7f92b86a5700 1 -- 172.21.15.123:0/3006568413 wait complete. 2023-10-06T04:32:42.029 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:32:42.035 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-10-06T04:32:40.902595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395196,"kb_used_data":16864,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752579524,"statfs":{"total":773094113280,"available":770641432576,"internally_reserved":0,"allocated":17268736,"data_stored":15231248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6289150,"num_objects":-20,"num_object_clones":0,"num_object_copies":-40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-20,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":51,"num_write_kb":6167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001036"},"pg_stats":[{"pgid":"6.3f","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270288+0000","last_change":"2023-10-06T04:32:24.268479+0000","last_active":"2023-10-06T04:32:34.270288+0000","last_peered":"2023-10-06T04:32:34.270288+0000","last_clean":"2023-10-06T04:32:34.270288+0000","last_became_active":"2023-10-06T04:32:24.268175+0000","last_became_peered":"2023-10-06T04:32:24.268175+0000","last_unstale":"2023-10-06T04:32:34.270288+0000","last_undegraded":"2023-10-06T04:32:34.270288+0000","last_fullsized":"2023-10-06T04:32:34.270288+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"34'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270143+0000","last_change":"2023-10-06T04:32:24.269331+0000","last_active":"2023-10-06T04:32:34.270143+0000","last_peered":"2023-10-06T04:32:34.270143+0000","last_clean":"2023-10-06T04:32:34.270143+0000","last_became_active":"2023-10-06T04:32:24.269096+0000","last_became_peered":"2023-10-06T04:32:24.269096+0000","last_unstale":"2023-10-06T04:32:34.270143+0000","last_undegraded":"2023-10-06T04:32:34.270143+0000","last_fullsized":"2023-10-06T04:32:34.270143+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270084+0000","last_change":"2023-10-06T04:32:24.267252+0000","last_active":"2023-10-06T04:32:34.270084+0000","last_peered":"2023-10-06T04:32:34.270084+0000","last_clean":"2023-10-06T04:32:34.270084+0000","last_became_active":"2023-10-06T04:32:24.266928+0000","last_became_peered":"2023-10-06T04:32:24.266928+0000","last_unstale":"2023-10-06T04:32:34.270084+0000","last_undegraded":"2023-10-06T04:32:34.270084+0000","last_fullsized":"2023-10-06T04:32:34.270084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263072+0000","last_change":"2023-10-06T04:32:24.271555+0000","last_active":"2023-10-06T04:32:34.263072+0000","last_peered":"2023-10-06T04:32:34.263072+0000","last_clean":"2023-10-06T04:32:34.263072+0000","last_became_active":"2023-10-06T04:32:24.271271+0000","last_became_peered":"2023-10-06T04:32:24.271271+0000","last_unstale":"2023-10-06T04:32:34.263072+0000","last_undegraded":"2023-10-06T04:32:34.263072+0000","last_fullsized":"2023-10-06T04:32:34.263072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"34'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263100+0000","last_change":"2023-10-06T04:32:24.270442+0000","last_active":"2023-10-06T04:32:34.263100+0000","last_peered":"2023-10-06T04:32:34.263100+0000","last_clean":"2023-10-06T04:32:34.263100+0000","last_became_active":"2023-10-06T04:32:24.270131+0000","last_became_peered":"2023-10-06T04:32:24.270131+0000","last_unstale":"2023-10-06T04:32:34.263100+0000","last_undegraded":"2023-10-06T04:32:34.263100+0000","last_fullsized":"2023-10-06T04:32:34.263100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268374+0000","last_change":"2023-10-06T04:32:24.263272+0000","last_active":"2023-10-06T04:32:34.268374+0000","last_peered":"2023-10-06T04:32:34.268374+0000","last_clean":"2023-10-06T04:32:34.268374+0000","last_became_active":"2023-10-06T04:32:24.262984+0000","last_became_peered":"2023-10-06T04:32:24.262984+0000","last_unstale":"2023-10-06T04:32:34.268374+0000","last_undegraded":"2023-10-06T04:32:34.268374+0000","last_fullsized":"2023-10-06T04:32:34.268374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:48.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315314+0000","last_change":"2023-10-06T04:32:24.263913+0000","last_active":"2023-10-06T04:32:35.315314+0000","last_peered":"2023-10-06T04:32:35.315314+0000","last_clean":"2023-10-06T04:32:35.315314+0000","last_became_active":"2023-10-06T04:32:24.263358+0000","last_became_peered":"2023-10-06T04:32:24.263358+0000","last_unstale":"2023-10-06T04:32:35.315314+0000","last_undegraded":"2023-10-06T04:32:35.315314+0000","last_fullsized":"2023-10-06T04:32:35.315314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269238+0000","last_change":"2023-10-06T04:32:24.266865+0000","last_active":"2023-10-06T04:32:34.269238+0000","last_peered":"2023-10-06T04:32:34.269238+0000","last_clean":"2023-10-06T04:32:34.269238+0000","last_became_active":"2023-10-06T04:32:24.266456+0000","last_became_peered":"2023-10-06T04:32:24.266456+0000","last_unstale":"2023-10-06T04:32:34.269238+0000","last_undegraded":"2023-10-06T04:32:34.269238+0000","last_fullsized":"2023-10-06T04:32:34.269238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"34'11","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270525+0000","last_change":"2023-10-06T04:32:24.267567+0000","last_active":"2023-10-06T04:32:34.270525+0000","last_peered":"2023-10-06T04:32:34.270525+0000","last_clean":"2023-10-06T04:32:34.270525+0000","last_became_active":"2023-10-06T04:32:24.267326+0000","last_became_peered":"2023-10-06T04:32:24.267326+0000","last_unstale":"2023-10-06T04:32:34.270525+0000","last_undegraded":"2023-10-06T04:32:34.270525+0000","last_fullsized":"2023-10-06T04:32:34.270525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"34'15","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315604+0000","last_change":"2023-10-06T04:32:24.266439+0000","last_active":"2023-10-06T04:32:35.315604+0000","last_peered":"2023-10-06T04:32:35.315604+0000","last_clean":"2023-10-06T04:32:35.315604+0000","last_became_active":"2023-10-06T04:32:24.266172+0000","last_became_peered":"2023-10-06T04:32:24.266172+0000","last_unstale":"2023-10-06T04:32:35.315604+0000","last_undegraded":"2023-10-06T04:32:35.315604+0000","last_fullsized":"2023-10-06T04:32:35.315604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:07.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"34'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315594+0000","last_change":"2023-10-06T04:32:24.264976+0000","last_active":"2023-10-06T04:32:35.315594+0000","last_peered":"2023-10-06T04:32:35.315594+0000","last_clean":"2023-10-06T04:32:35.315594+0000","last_became_active":"2023-10-06T04:32:24.264647+0000","last_became_peered":"2023-10-06T04:32:24.264647+0000","last_unstale":"2023-10-06T04:32:35.315594+0000","last_undegraded":"2023-10-06T04:32:35.315594+0000","last_fullsized":"2023-10-06T04:32:35.315594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269897+0000","last_change":"2023-10-06T04:32:24.262121+0000","last_active":"2023-10-06T04:32:34.269897+0000","last_peered":"2023-10-06T04:32:34.269897+0000","last_clean":"2023-10-06T04:32:34.269897+0000","last_became_active":"2023-10-06T04:32:24.261846+0000","last_became_peered":"2023-10-06T04:32:24.261846+0000","last_unstale":"2023-10-06T04:32:34.269897+0000","last_undegraded":"2023-10-06T04:32:34.269897+0000","last_fullsized":"2023-10-06T04:32:34.269897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315289+0000","last_change":"2023-10-06T04:32:24.267116+0000","last_active":"2023-10-06T04:32:35.315289+0000","last_peered":"2023-10-06T04:32:35.315289+0000","last_clean":"2023-10-06T04:32:35.315289+0000","last_became_active":"2023-10-06T04:32:24.266756+0000","last_became_peered":"2023-10-06T04:32:24.266756+0000","last_unstale":"2023-10-06T04:32:35.315289+0000","last_undegraded":"2023-10-06T04:32:35.315289+0000","last_fullsized":"2023-10-06T04:32:35.315289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:26.457033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261300+0000","last_change":"2023-10-06T04:32:24.267554+0000","last_active":"2023-10-06T04:32:34.261300+0000","last_peered":"2023-10-06T04:32:34.261300+0000","last_clean":"2023-10-06T04:32:34.261300+0000","last_became_active":"2023-10-06T04:32:24.266198+0000","last_became_peered":"2023-10-06T04:32:24.266198+0000","last_unstale":"2023-10-06T04:32:34.261300+0000","last_undegraded":"2023-10-06T04:32:34.261300+0000","last_fullsized":"2023-10-06T04:32:34.261300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315261+0000","last_change":"2023-10-06T04:32:24.265234+0000","last_active":"2023-10-06T04:32:35.315261+0000","last_peered":"2023-10-06T04:32:35.315261+0000","last_clean":"2023-10-06T04:32:35.315261+0000","last_became_active":"2023-10-06T04:32:24.264892+0000","last_became_peered":"2023-10-06T04:32:24.264892+0000","last_unstale":"2023-10-06T04:32:35.315261+0000","last_undegraded":"2023-10-06T04:32:35.315261+0000","last_fullsized":"2023-10-06T04:32:35.315261+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268527+0000","last_change":"2023-10-06T04:32:24.268696+0000","last_active":"2023-10-06T04:32:34.268527+0000","last_peered":"2023-10-06T04:32:34.268527+0000","last_clean":"2023-10-06T04:32:34.268527+0000","last_became_active":"2023-10-06T04:32:24.268430+0000","last_became_peered":"2023-10-06T04:32:24.268430+0000","last_unstale":"2023-10-06T04:32:34.268527+0000","last_undegraded":"2023-10-06T04:32:34.268527+0000","last_fullsized":"2023-10-06T04:32:34.268527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268528+0000","last_change":"2023-10-06T04:32:24.269272+0000","last_active":"2023-10-06T04:32:34.268528+0000","last_peered":"2023-10-06T04:32:34.268528+0000","last_clean":"2023-10-06T04:32:34.268528+0000","last_became_active":"2023-10-06T04:32:24.268793+0000","last_became_peered":"2023-10-06T04:32:24.268793+0000","last_unstale":"2023-10-06T04:32:34.268528+0000","last_undegraded":"2023-10-06T04:32:34.268528+0000","last_fullsized":"2023-10-06T04:32:34.268528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"34'10","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268890+0000","last_change":"2023-10-06T04:32:24.268089+0000","last_active":"2023-10-06T04:32:34.268890+0000","last_peered":"2023-10-06T04:32:34.268890+0000","last_clean":"2023-10-06T04:32:34.268890+0000","last_became_active":"2023-10-06T04:32:24.267842+0000","last_became_peered":"2023-10-06T04:32:24.267842+0000","last_unstale":"2023-10-06T04:32:34.268890+0000","last_undegraded":"2023-10-06T04:32:34.268890+0000","last_fullsized":"2023-10-06T04:32:34.268890+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:36.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"34'7","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263103+0000","last_change":"2023-10-06T04:32:24.263185+0000","last_active":"2023-10-06T04:32:34.263103+0000","last_peered":"2023-10-06T04:32:34.263103+0000","last_clean":"2023-10-06T04:32:34.263103+0000","last_became_active":"2023-10-06T04:32:24.262500+0000","last_became_peered":"2023-10-06T04:32:24.262500+0000","last_unstale":"2023-10-06T04:32:34.263103+0000","last_undegraded":"2023-10-06T04:32:34.263103+0000","last_fullsized":"2023-10-06T04:32:34.263103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:18.096304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268374+0000","last_change":"2023-10-06T04:32:24.266364+0000","last_active":"2023-10-06T04:32:34.268374+0000","last_peered":"2023-10-06T04:32:34.268374+0000","last_clean":"2023-10-06T04:32:34.268374+0000","last_became_active":"2023-10-06T04:32:24.266082+0000","last_became_peered":"2023-10-06T04:32:24.266082+0000","last_unstale":"2023-10-06T04:32:34.268374+0000","last_undegraded":"2023-10-06T04:32:34.268374+0000","last_fullsized":"2023-10-06T04:32:34.268374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"34'11","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315320+0000","last_change":"2023-10-06T04:32:24.265143+0000","last_active":"2023-10-06T04:32:35.315320+0000","last_peered":"2023-10-06T04:32:35.315320+0000","last_clean":"2023-10-06T04:32:35.315320+0000","last_became_active":"2023-10-06T04:32:24.264481+0000","last_became_peered":"2023-10-06T04:32:24.264481+0000","last_unstale":"2023-10-06T04:32:35.315320+0000","last_undegraded":"2023-10-06T04:32:35.315320+0000","last_fullsized":"2023-10-06T04:32:35.315320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:04.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260770+0000","last_change":"2023-10-06T04:32:24.262815+0000","last_active":"2023-10-06T04:32:34.260770+0000","last_peered":"2023-10-06T04:32:34.260770+0000","last_clean":"2023-10-06T04:32:34.260770+0000","last_became_active":"2023-10-06T04:32:24.262576+0000","last_became_peered":"2023-10-06T04:32:24.262576+0000","last_unstale":"2023-10-06T04:32:34.260770+0000","last_undegraded":"2023-10-06T04:32:34.260770+0000","last_fullsized":"2023-10-06T04:32:34.260770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"34'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269183+0000","last_change":"2023-10-06T04:32:24.269201+0000","last_active":"2023-10-06T04:32:34.269183+0000","last_peered":"2023-10-06T04:32:34.269183+0000","last_clean":"2023-10-06T04:32:34.269183+0000","last_became_active":"2023-10-06T04:32:24.268841+0000","last_became_peered":"2023-10-06T04:32:24.268841+0000","last_unstale":"2023-10-06T04:32:34.269183+0000","last_undegraded":"2023-10-06T04:32:34.269183+0000","last_fullsized":"2023-10-06T04:32:34.269183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"34'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262874+0000","last_change":"2023-10-06T04:32:24.264875+0000","last_active":"2023-10-06T04:32:34.262874+0000","last_peered":"2023-10-06T04:32:34.262874+0000","last_clean":"2023-10-06T04:32:34.262874+0000","last_became_active":"2023-10-06T04:32:24.264601+0000","last_became_peered":"2023-10-06T04:32:24.264601+0000","last_unstale":"2023-10-06T04:32:34.262874+0000","last_undegraded":"2023-10-06T04:32:34.262874+0000","last_fullsized":"2023-10-06T04:32:34.262874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:38.680988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"34'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269157+0000","last_change":"2023-10-06T04:32:24.268714+0000","last_active":"2023-10-06T04:32:34.269157+0000","last_peered":"2023-10-06T04:32:34.269157+0000","last_clean":"2023-10-06T04:32:34.269157+0000","last_became_active":"2023-10-06T04:32:24.268440+0000","last_became_peered":"2023-10-06T04:32:24.268440+0000","last_unstale":"2023-10-06T04:32:34.269157+0000","last_undegraded":"2023-10-06T04:32:34.269157+0000","last_fullsized":"2023-10-06T04:32:34.269157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:24.913639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268884+0000","last_change":"2023-10-06T04:32:24.266945+0000","last_active":"2023-10-06T04:32:34.268884+0000","last_peered":"2023-10-06T04:32:34.268884+0000","last_clean":"2023-10-06T04:32:34.268884+0000","last_became_active":"2023-10-06T04:32:24.266673+0000","last_became_peered":"2023-10-06T04:32:24.266673+0000","last_unstale":"2023-10-06T04:32:34.268884+0000","last_undegraded":"2023-10-06T04:32:34.268884+0000","last_fullsized":"2023-10-06T04:32:34.268884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"34'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268528+0000","last_change":"2023-10-06T04:32:24.270531+0000","last_active":"2023-10-06T04:32:34.268528+0000","last_peered":"2023-10-06T04:32:34.268528+0000","last_clean":"2023-10-06T04:32:34.268528+0000","last_became_active":"2023-10-06T04:32:24.270268+0000","last_became_peered":"2023-10-06T04:32:24.270268+0000","last_unstale":"2023-10-06T04:32:34.268528+0000","last_undegraded":"2023-10-06T04:32:34.268528+0000","last_fullsized":"2023-10-06T04:32:34.268528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:34.859497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"34'4","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315561+0000","last_change":"2023-10-06T04:32:24.265317+0000","last_active":"2023-10-06T04:32:35.315561+0000","last_peered":"2023-10-06T04:32:35.315561+0000","last_clean":"2023-10-06T04:32:35.315561+0000","last_became_active":"2023-10-06T04:32:24.264377+0000","last_became_peered":"2023-10-06T04:32:24.264377+0000","last_unstale":"2023-10-06T04:32:35.315561+0000","last_undegraded":"2023-10-06T04:32:35.315561+0000","last_fullsized":"2023-10-06T04:32:35.315561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"34'7","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268365+0000","last_change":"2023-10-06T04:32:24.264442+0000","last_active":"2023-10-06T04:32:34.268365+0000","last_peered":"2023-10-06T04:32:34.268365+0000","last_clean":"2023-10-06T04:32:34.268365+0000","last_became_active":"2023-10-06T04:32:24.264148+0000","last_became_peered":"2023-10-06T04:32:24.264148+0000","last_unstale":"2023-10-06T04:32:34.268365+0000","last_undegraded":"2023-10-06T04:32:34.268365+0000","last_fullsized":"2023-10-06T04:32:34.268365+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268651+0000","last_change":"2023-10-06T04:32:24.270800+0000","last_active":"2023-10-06T04:32:34.268651+0000","last_peered":"2023-10-06T04:32:34.268651+0000","last_clean":"2023-10-06T04:32:34.268651+0000","last_became_active":"2023-10-06T04:32:24.270560+0000","last_became_peered":"2023-10-06T04:32:24.270560+0000","last_unstale":"2023-10-06T04:32:34.268651+0000","last_undegraded":"2023-10-06T04:32:34.268651+0000","last_fullsized":"2023-10-06T04:32:34.268651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"34'12","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269073+0000","last_change":"2023-10-06T04:32:24.271609+0000","last_active":"2023-10-06T04:32:34.269073+0000","last_peered":"2023-10-06T04:32:34.269073+0000","last_clean":"2023-10-06T04:32:34.269073+0000","last_became_active":"2023-10-06T04:32:24.271354+0000","last_became_peered":"2023-10-06T04:32:24.271354+0000","last_unstale":"2023-10-06T04:32:34.269073+0000","last_undegraded":"2023-10-06T04:32:34.269073+0000","last_fullsized":"2023-10-06T04:32:34.269073+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270603+0000","last_change":"2023-10-06T04:32:24.263719+0000","last_active":"2023-10-06T04:32:34.270603+0000","last_peered":"2023-10-06T04:32:34.270603+0000","last_clean":"2023-10-06T04:32:34.270603+0000","last_became_active":"2023-10-06T04:32:24.263442+0000","last_became_peered":"2023-10-06T04:32:24.263442+0000","last_unstale":"2023-10-06T04:32:34.270603+0000","last_undegraded":"2023-10-06T04:32:34.270603+0000","last_fullsized":"2023-10-06T04:32:34.270603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:40.811547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268733+0000","last_change":"2023-10-06T04:32:24.270093+0000","last_active":"2023-10-06T04:32:34.268733+0000","last_peered":"2023-10-06T04:32:34.268733+0000","last_clean":"2023-10-06T04:32:34.268733+0000","last_became_active":"2023-10-06T04:32:24.269851+0000","last_became_peered":"2023-10-06T04:32:24.269851+0000","last_unstale":"2023-10-06T04:32:34.268733+0000","last_undegraded":"2023-10-06T04:32:34.268733+0000","last_fullsized":"2023-10-06T04:32:34.268733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'8","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269249+0000","last_change":"2023-10-06T04:32:23.441194+0000","last_active":"2023-10-06T04:32:34.269249+0000","last_peered":"2023-10-06T04:32:34.269249+0000","last_clean":"2023-10-06T04:32:34.269249+0000","last_became_active":"2023-10-06T04:32:23.440849+0000","last_became_peered":"2023-10-06T04:32:23.440849+0000","last_unstale":"2023-10-06T04:32:34.269249+0000","last_undegraded":"2023-10-06T04:32:34.269249+0000","last_fullsized":"2023-10-06T04:32:34.269249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:59.829363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268727+0000","last_change":"2023-10-06T04:32:24.267279+0000","last_active":"2023-10-06T04:32:34.268727+0000","last_peered":"2023-10-06T04:32:34.268727+0000","last_clean":"2023-10-06T04:32:34.268727+0000","last_became_active":"2023-10-06T04:32:24.267016+0000","last_became_peered":"2023-10-06T04:32:24.267016+0000","last_unstale":"2023-10-06T04:32:34.268727+0000","last_undegraded":"2023-10-06T04:32:34.268727+0000","last_fullsized":"2023-10-06T04:32:34.268727+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268798+0000","last_change":"2023-10-06T04:32:23.450031+0000","last_active":"2023-10-06T04:32:34.268798+0000","last_peered":"2023-10-06T04:32:34.268798+0000","last_clean":"2023-10-06T04:32:34.268798+0000","last_became_active":"2023-10-06T04:32:23.449717+0000","last_became_peered":"2023-10-06T04:32:23.449717+0000","last_unstale":"2023-10-06T04:32:34.268798+0000","last_undegraded":"2023-10-06T04:32:34.268798+0000","last_fullsized":"2023-10-06T04:32:34.268798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:19.596388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260810+0000","last_change":"2023-10-06T04:32:24.267302+0000","last_active":"2023-10-06T04:32:34.260810+0000","last_peered":"2023-10-06T04:32:34.260810+0000","last_clean":"2023-10-06T04:32:34.260810+0000","last_became_active":"2023-10-06T04:32:24.266976+0000","last_became_peered":"2023-10-06T04:32:24.266976+0000","last_unstale":"2023-10-06T04:32:34.260810+0000","last_undegraded":"2023-10-06T04:32:34.260810+0000","last_fullsized":"2023-10-06T04:32:34.260810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268890+0000","last_change":"2023-10-06T04:32:23.447029+0000","last_active":"2023-10-06T04:32:34.268890+0000","last_peered":"2023-10-06T04:32:34.268890+0000","last_clean":"2023-10-06T04:32:34.268890+0000","last_became_active":"2023-10-06T04:32:23.446659+0000","last_became_peered":"2023-10-06T04:32:23.446659+0000","last_unstale":"2023-10-06T04:32:34.268890+0000","last_undegraded":"2023-10-06T04:32:34.268890+0000","last_fullsized":"2023-10-06T04:32:34.268890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:13.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270340+0000","last_change":"2023-10-06T04:32:24.265927+0000","last_active":"2023-10-06T04:32:34.270340+0000","last_peered":"2023-10-06T04:32:34.270340+0000","last_clean":"2023-10-06T04:32:34.270340+0000","last_became_active":"2023-10-06T04:32:24.265668+0000","last_became_peered":"2023-10-06T04:32:24.265668+0000","last_unstale":"2023-10-06T04:32:34.270340+0000","last_undegraded":"2023-10-06T04:32:34.270340+0000","last_fullsized":"2023-10-06T04:32:34.270340+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260994+0000","last_change":"2023-10-06T04:32:23.437810+0000","last_active":"2023-10-06T04:32:34.260994+0000","last_peered":"2023-10-06T04:32:34.260994+0000","last_clean":"2023-10-06T04:32:34.260994+0000","last_became_active":"2023-10-06T04:32:23.437533+0000","last_became_peered":"2023-10-06T04:32:23.437533+0000","last_unstale":"2023-10-06T04:32:34.260994+0000","last_undegraded":"2023-10-06T04:32:34.260994+0000","last_fullsized":"2023-10-06T04:32:34.260994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:46.149228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268666+0000","last_change":"2023-10-06T04:32:24.261934+0000","last_active":"2023-10-06T04:32:34.268666+0000","last_peered":"2023-10-06T04:32:34.268666+0000","last_clean":"2023-10-06T04:32:34.268666+0000","last_became_active":"2023-10-06T04:32:24.261681+0000","last_became_peered":"2023-10-06T04:32:24.261681+0000","last_unstale":"2023-10-06T04:32:34.268666+0000","last_undegraded":"2023-10-06T04:32:34.268666+0000","last_fullsized":"2023-10-06T04:32:34.268666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315427+0000","last_change":"2023-10-06T04:32:23.443656+0000","last_active":"2023-10-06T04:32:35.315427+0000","last_peered":"2023-10-06T04:32:35.315427+0000","last_clean":"2023-10-06T04:32:35.315427+0000","last_became_active":"2023-10-06T04:32:23.443293+0000","last_became_peered":"2023-10-06T04:32:23.443293+0000","last_unstale":"2023-10-06T04:32:35.315427+0000","last_undegraded":"2023-10-06T04:32:35.315427+0000","last_fullsized":"2023-10-06T04:32:35.315427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315387+0000","last_change":"2023-10-06T04:32:24.264104+0000","last_active":"2023-10-06T04:32:35.315387+0000","last_peered":"2023-10-06T04:32:35.315387+0000","last_clean":"2023-10-06T04:32:35.315387+0000","last_became_active":"2023-10-06T04:32:24.263651+0000","last_became_peered":"2023-10-06T04:32:24.263651+0000","last_unstale":"2023-10-06T04:32:35.315387+0000","last_undegraded":"2023-10-06T04:32:35.315387+0000","last_fullsized":"2023-10-06T04:32:35.315387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:24.395312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269261+0000","last_change":"2023-10-06T04:32:23.451767+0000","last_active":"2023-10-06T04:32:34.269261+0000","last_peered":"2023-10-06T04:32:34.269261+0000","last_clean":"2023-10-06T04:32:34.269261+0000","last_became_active":"2023-10-06T04:32:23.451430+0000","last_became_peered":"2023-10-06T04:32:23.451430+0000","last_unstale":"2023-10-06T04:32:34.269261+0000","last_undegraded":"2023-10-06T04:32:34.269261+0000","last_fullsized":"2023-10-06T04:32:34.269261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261339+0000","last_change":"2023-10-06T04:32:24.267750+0000","last_active":"2023-10-06T04:32:34.261339+0000","last_peered":"2023-10-06T04:32:34.261339+0000","last_clean":"2023-10-06T04:32:34.261339+0000","last_became_active":"2023-10-06T04:32:24.265805+0000","last_became_peered":"2023-10-06T04:32:24.265805+0000","last_unstale":"2023-10-06T04:32:34.261339+0000","last_undegraded":"2023-10-06T04:32:34.261339+0000","last_fullsized":"2023-10-06T04:32:34.261339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268431+0000","last_change":"2023-10-06T04:32:23.449278+0000","last_active":"2023-10-06T04:32:34.268431+0000","last_peered":"2023-10-06T04:32:34.268431+0000","last_clean":"2023-10-06T04:32:34.268431+0000","last_became_active":"2023-10-06T04:32:23.449034+0000","last_became_peered":"2023-10-06T04:32:23.449034+0000","last_unstale":"2023-10-06T04:32:34.268431+0000","last_undegraded":"2023-10-06T04:32:34.268431+0000","last_fullsized":"2023-10-06T04:32:34.268431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269734+0000","last_change":"2023-10-06T04:32:24.265155+0000","last_active":"2023-10-06T04:32:34.269734+0000","last_peered":"2023-10-06T04:32:34.269734+0000","last_clean":"2023-10-06T04:32:34.269734+0000","last_became_active":"2023-10-06T04:32:24.264863+0000","last_became_peered":"2023-10-06T04:32:24.264863+0000","last_unstale":"2023-10-06T04:32:34.269734+0000","last_undegraded":"2023-10-06T04:32:34.269734+0000","last_fullsized":"2023-10-06T04:32:34.269734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260606+0000","last_change":"2023-10-06T04:32:23.440295+0000","last_active":"2023-10-06T04:32:34.260606+0000","last_peered":"2023-10-06T04:32:34.260606+0000","last_clean":"2023-10-06T04:32:34.260606+0000","last_became_active":"2023-10-06T04:32:23.440039+0000","last_became_peered":"2023-10-06T04:32:23.440039+0000","last_unstale":"2023-10-06T04:32:34.260606+0000","last_undegraded":"2023-10-06T04:32:34.260606+0000","last_fullsized":"2023-10-06T04:32:34.260606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"34'11","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263151+0000","last_change":"2023-10-06T04:32:24.271227+0000","last_active":"2023-10-06T04:32:34.263151+0000","last_peered":"2023-10-06T04:32:34.263151+0000","last_clean":"2023-10-06T04:32:34.263151+0000","last_became_active":"2023-10-06T04:32:24.270984+0000","last_became_peered":"2023-10-06T04:32:24.270984+0000","last_unstale":"2023-10-06T04:32:34.263151+0000","last_undegraded":"2023-10-06T04:32:34.263151+0000","last_fullsized":"2023-10-06T04:32:34.263151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270397+0000","last_change":"2023-10-06T04:32:23.447791+0000","last_active":"2023-10-06T04:32:34.270397+0000","last_peered":"2023-10-06T04:32:34.270397+0000","last_clean":"2023-10-06T04:32:34.270397+0000","last_became_active":"2023-10-06T04:32:23.447395+0000","last_became_peered":"2023-10-06T04:32:23.447395+0000","last_unstale":"2023-10-06T04:32:34.270397+0000","last_undegraded":"2023-10-06T04:32:34.270397+0000","last_fullsized":"2023-10-06T04:32:34.270397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:18.839316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"34'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269473+0000","last_change":"2023-10-06T04:32:24.269565+0000","last_active":"2023-10-06T04:32:34.269473+0000","last_peered":"2023-10-06T04:32:34.269473+0000","last_clean":"2023-10-06T04:32:34.269473+0000","last_became_active":"2023-10-06T04:32:24.269285+0000","last_became_peered":"2023-10-06T04:32:24.269285+0000","last_unstale":"2023-10-06T04:32:34.269473+0000","last_undegraded":"2023-10-06T04:32:34.269473+0000","last_fullsized":"2023-10-06T04:32:34.269473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315749+0000","last_change":"2023-10-06T04:32:23.444329+0000","last_active":"2023-10-06T04:32:35.315749+0000","last_peered":"2023-10-06T04:32:35.315749+0000","last_clean":"2023-10-06T04:32:35.315749+0000","last_became_active":"2023-10-06T04:32:23.444020+0000","last_became_peered":"2023-10-06T04:32:23.444020+0000","last_unstale":"2023-10-06T04:32:35.315749+0000","last_undegraded":"2023-10-06T04:32:35.315749+0000","last_fullsized":"2023-10-06T04:32:35.315749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"34'10","reported_seq":22,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268693+0000","last_change":"2023-10-06T04:32:24.264060+0000","last_active":"2023-10-06T04:32:34.268693+0000","last_peered":"2023-10-06T04:32:34.268693+0000","last_clean":"2023-10-06T04:32:34.268693+0000","last_became_active":"2023-10-06T04:32:24.263825+0000","last_became_peered":"2023-10-06T04:32:24.263825+0000","last_unstale":"2023-10-06T04:32:34.268693+0000","last_undegraded":"2023-10-06T04:32:34.268693+0000","last_fullsized":"2023-10-06T04:32:34.268693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'5","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315712+0000","last_change":"2023-10-06T04:32:23.447887+0000","last_active":"2023-10-06T04:32:35.315712+0000","last_peered":"2023-10-06T04:32:35.315712+0000","last_clean":"2023-10-06T04:32:35.315712+0000","last_became_active":"2023-10-06T04:32:23.447650+0000","last_became_peered":"2023-10-06T04:32:23.447650+0000","last_unstale":"2023-10-06T04:32:35.315712+0000","last_undegraded":"2023-10-06T04:32:35.315712+0000","last_fullsized":"2023-10-06T04:32:35.315712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:47.164678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260931+0000","last_change":"2023-10-06T04:32:24.270050+0000","last_active":"2023-10-06T04:32:34.260931+0000","last_peered":"2023-10-06T04:32:34.260931+0000","last_clean":"2023-10-06T04:32:34.260931+0000","last_became_active":"2023-10-06T04:32:24.269298+0000","last_became_peered":"2023-10-06T04:32:24.269298+0000","last_unstale":"2023-10-06T04:32:34.260931+0000","last_undegraded":"2023-10-06T04:32:34.260931+0000","last_fullsized":"2023-10-06T04:32:34.260931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'22","reported_seq":35,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269989+0000","last_change":"2023-10-06T04:32:23.447643+0000","last_active":"2023-10-06T04:32:34.269989+0000","last_peered":"2023-10-06T04:32:34.269989+0000","last_clean":"2023-10-06T04:32:34.269989+0000","last_became_active":"2023-10-06T04:32:23.447352+0000","last_became_peered":"2023-10-06T04:32:23.447352+0000","last_unstale":"2023-10-06T04:32:34.269989+0000","last_undegraded":"2023-10-06T04:32:34.269989+0000","last_fullsized":"2023-10-06T04:32:34.269989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268957+0000","last_change":"2023-10-06T04:32:24.267086+0000","last_active":"2023-10-06T04:32:34.268957+0000","last_peered":"2023-10-06T04:32:34.268957+0000","last_clean":"2023-10-06T04:32:34.268957+0000","last_became_active":"2023-10-06T04:32:24.266829+0000","last_became_peered":"2023-10-06T04:32:24.266829+0000","last_unstale":"2023-10-06T04:32:34.268957+0000","last_undegraded":"2023-10-06T04:32:34.268957+0000","last_fullsized":"2023-10-06T04:32:34.268957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'5","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269791+0000","last_change":"2023-10-06T04:32:23.443222+0000","last_active":"2023-10-06T04:32:34.269791+0000","last_peered":"2023-10-06T04:32:34.269791+0000","last_clean":"2023-10-06T04:32:34.269791+0000","last_became_active":"2023-10-06T04:32:23.442957+0000","last_became_peered":"2023-10-06T04:32:23.442957+0000","last_unstale":"2023-10-06T04:32:34.269791+0000","last_undegraded":"2023-10-06T04:32:34.269791+0000","last_fullsized":"2023-10-06T04:32:34.269791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269309+0000","last_change":"2023-10-06T04:32:24.268660+0000","last_active":"2023-10-06T04:32:34.269309+0000","last_peered":"2023-10-06T04:32:34.269309+0000","last_clean":"2023-10-06T04:32:34.269309+0000","last_became_active":"2023-10-06T04:32:24.268397+0000","last_became_peered":"2023-10-06T04:32:24.268397+0000","last_unstale":"2023-10-06T04:32:34.269309+0000","last_undegraded":"2023-10-06T04:32:34.269309+0000","last_fullsized":"2023-10-06T04:32:34.269309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268494+0000","last_change":"2023-10-06T04:32:23.445203+0000","last_active":"2023-10-06T04:32:34.268494+0000","last_peered":"2023-10-06T04:32:34.268494+0000","last_clean":"2023-10-06T04:32:34.268494+0000","last_became_active":"2023-10-06T04:32:23.444688+0000","last_became_peered":"2023-10-06T04:32:23.444688+0000","last_unstale":"2023-10-06T04:32:34.268494+0000","last_undegraded":"2023-10-06T04:32:34.268494+0000","last_fullsized":"2023-10-06T04:32:34.268494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269146+0000","last_change":"2023-10-06T04:32:24.271729+0000","last_active":"2023-10-06T04:32:34.269146+0000","last_peered":"2023-10-06T04:32:34.269146+0000","last_clean":"2023-10-06T04:32:34.269146+0000","last_became_active":"2023-10-06T04:32:24.271453+0000","last_became_peered":"2023-10-06T04:32:24.271453+0000","last_unstale":"2023-10-06T04:32:34.269146+0000","last_undegraded":"2023-10-06T04:32:34.269146+0000","last_fullsized":"2023-10-06T04:32:34.269146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268204+0000","last_change":"2023-10-06T04:32:23.440530+0000","last_active":"2023-10-06T04:32:34.268204+0000","last_peered":"2023-10-06T04:32:34.268204+0000","last_clean":"2023-10-06T04:32:34.268204+0000","last_became_active":"2023-10-06T04:32:23.440226+0000","last_became_peered":"2023-10-06T04:32:23.440226+0000","last_unstale":"2023-10-06T04:32:34.268204+0000","last_undegraded":"2023-10-06T04:32:34.268204+0000","last_fullsized":"2023-10-06T04:32:34.268204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269621+0000","last_change":"2023-10-06T04:32:24.268363+0000","last_active":"2023-10-06T04:32:34.269621+0000","last_peered":"2023-10-06T04:32:34.269621+0000","last_clean":"2023-10-06T04:32:34.269621+0000","last_became_active":"2023-10-06T04:32:24.268116+0000","last_became_peered":"2023-10-06T04:32:24.268116+0000","last_unstale":"2023-10-06T04:32:34.269621+0000","last_undegraded":"2023-10-06T04:32:34.269621+0000","last_fullsized":"2023-10-06T04:32:34.269621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'15","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268764+0000","last_change":"2023-10-06T04:32:23.452500+0000","last_active":"2023-10-06T04:32:34.268764+0000","last_peered":"2023-10-06T04:32:34.268764+0000","last_clean":"2023-10-06T04:32:34.268764+0000","last_became_active":"2023-10-06T04:32:23.452236+0000","last_became_peered":"2023-10-06T04:32:23.452236+0000","last_unstale":"2023-10-06T04:32:34.268764+0000","last_undegraded":"2023-10-06T04:32:34.268764+0000","last_fullsized":"2023-10-06T04:32:34.268764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269934+0000","last_change":"2023-10-06T04:32:24.265984+0000","last_active":"2023-10-06T04:32:34.269934+0000","last_peered":"2023-10-06T04:32:34.269934+0000","last_clean":"2023-10-06T04:32:34.269934+0000","last_became_active":"2023-10-06T04:32:24.265707+0000","last_became_peered":"2023-10-06T04:32:24.265707+0000","last_unstale":"2023-10-06T04:32:34.269934+0000","last_undegraded":"2023-10-06T04:32:34.269934+0000","last_fullsized":"2023-10-06T04:32:34.269934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315804+0000","last_change":"2023-10-06T04:32:23.447020+0000","last_active":"2023-10-06T04:32:35.315804+0000","last_peered":"2023-10-06T04:32:35.315804+0000","last_clean":"2023-10-06T04:32:35.315804+0000","last_became_active":"2023-10-06T04:32:23.446786+0000","last_became_peered":"2023-10-06T04:32:23.446786+0000","last_unstale":"2023-10-06T04:32:35.315804+0000","last_undegraded":"2023-10-06T04:32:35.315804+0000","last_fullsized":"2023-10-06T04:32:35.315804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.805674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315600+0000","last_change":"2023-10-06T04:32:24.265722+0000","last_active":"2023-10-06T04:32:35.315600+0000","last_peered":"2023-10-06T04:32:35.315600+0000","last_clean":"2023-10-06T04:32:35.315600+0000","last_became_active":"2023-10-06T04:32:24.265435+0000","last_became_peered":"2023-10-06T04:32:24.265435+0000","last_unstale":"2023-10-06T04:32:35.315600+0000","last_undegraded":"2023-10-06T04:32:35.315600+0000","last_fullsized":"2023-10-06T04:32:35.315600+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.261738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260867+0000","last_change":"2023-10-06T04:32:23.442348+0000","last_active":"2023-10-06T04:32:34.260867+0000","last_peered":"2023-10-06T04:32:34.260867+0000","last_clean":"2023-10-06T04:32:34.260867+0000","last_became_active":"2023-10-06T04:32:23.441833+0000","last_became_peered":"2023-10-06T04:32:23.441833+0000","last_unstale":"2023-10-06T04:32:34.260867+0000","last_undegraded":"2023-10-06T04:32:34.260867+0000","last_fullsized":"2023-10-06T04:32:34.260867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.916159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260744+0000","last_change":"2023-10-06T04:32:24.254622+0000","last_active":"2023-10-06T04:32:34.260744+0000","last_peered":"2023-10-06T04:32:34.260744+0000","last_clean":"2023-10-06T04:32:34.260744+0000","last_became_active":"2023-10-06T04:32:24.254294+0000","last_became_peered":"2023-10-06T04:32:24.254294+0000","last_unstale":"2023-10-06T04:32:34.260744+0000","last_undegraded":"2023-10-06T04:32:34.260744+0000","last_fullsized":"2023-10-06T04:32:34.260744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263152+0000","last_change":"2023-10-06T04:32:23.447777+0000","last_active":"2023-10-06T04:32:34.263152+0000","last_peered":"2023-10-06T04:32:34.263152+0000","last_clean":"2023-10-06T04:32:34.263152+0000","last_became_active":"2023-10-06T04:32:23.447526+0000","last_became_peered":"2023-10-06T04:32:23.447526+0000","last_unstale":"2023-10-06T04:32:34.263152+0000","last_undegraded":"2023-10-06T04:32:34.263152+0000","last_fullsized":"2023-10-06T04:32:34.263152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:59.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"34'9","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263255+0000","last_change":"2023-10-06T04:32:24.270632+0000","last_active":"2023-10-06T04:32:34.263255+0000","last_peered":"2023-10-06T04:32:34.263255+0000","last_clean":"2023-10-06T04:32:34.263255+0000","last_became_active":"2023-10-06T04:32:24.270263+0000","last_became_peered":"2023-10-06T04:32:24.270263+0000","last_unstale":"2023-10-06T04:32:34.263255+0000","last_undegraded":"2023-10-06T04:32:34.263255+0000","last_fullsized":"2023-10-06T04:32:34.263255+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:19.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268936+0000","last_change":"2023-10-06T04:32:23.446504+0000","last_active":"2023-10-06T04:32:34.268936+0000","last_peered":"2023-10-06T04:32:34.268936+0000","last_clean":"2023-10-06T04:32:34.268936+0000","last_became_active":"2023-10-06T04:32:23.446221+0000","last_became_peered":"2023-10-06T04:32:23.446221+0000","last_unstale":"2023-10-06T04:32:34.268936+0000","last_undegraded":"2023-10-06T04:32:34.268936+0000","last_fullsized":"2023-10-06T04:32:34.268936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268647+0000","last_change":"2023-10-06T04:32:24.271430+0000","last_active":"2023-10-06T04:32:34.268647+0000","last_peered":"2023-10-06T04:32:34.268647+0000","last_clean":"2023-10-06T04:32:34.268647+0000","last_became_active":"2023-10-06T04:32:24.271180+0000","last_became_peered":"2023-10-06T04:32:24.271180+0000","last_unstale":"2023-10-06T04:32:34.268647+0000","last_undegraded":"2023-10-06T04:32:34.268647+0000","last_fullsized":"2023-10-06T04:32:34.268647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'4","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263236+0000","last_change":"2023-10-06T04:32:23.445911+0000","last_active":"2023-10-06T04:32:34.263236+0000","last_peered":"2023-10-06T04:32:34.263236+0000","last_clean":"2023-10-06T04:32:34.263236+0000","last_became_active":"2023-10-06T04:32:23.445582+0000","last_became_peered":"2023-10-06T04:32:23.445582+0000","last_unstale":"2023-10-06T04:32:34.263236+0000","last_undegraded":"2023-10-06T04:32:34.263236+0000","last_fullsized":"2023-10-06T04:32:34.263236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:35.404671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"34'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269245+0000","last_change":"2023-10-06T04:32:24.259250+0000","last_active":"2023-10-06T04:32:34.269245+0000","last_peered":"2023-10-06T04:32:34.269245+0000","last_clean":"2023-10-06T04:32:34.269245+0000","last_became_active":"2023-10-06T04:32:24.258962+0000","last_became_peered":"2023-10-06T04:32:24.258962+0000","last_unstale":"2023-10-06T04:32:34.269245+0000","last_undegraded":"2023-10-06T04:32:34.269245+0000","last_fullsized":"2023-10-06T04:32:34.269245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262972+0000","last_change":"2023-10-06T04:32:23.445529+0000","last_active":"2023-10-06T04:32:34.262972+0000","last_peered":"2023-10-06T04:32:34.262972+0000","last_clean":"2023-10-06T04:32:34.262972+0000","last_became_active":"2023-10-06T04:32:23.445245+0000","last_became_peered":"2023-10-06T04:32:23.445245+0000","last_unstale":"2023-10-06T04:32:34.262972+0000","last_undegraded":"2023-10-06T04:32:34.262972+0000","last_fullsized":"2023-10-06T04:32:34.262972+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315277+0000","last_change":"2023-10-06T04:32:24.270658+0000","last_active":"2023-10-06T04:32:35.315277+0000","last_peered":"2023-10-06T04:32:35.315277+0000","last_clean":"2023-10-06T04:32:35.315277+0000","last_became_active":"2023-10-06T04:32:24.270394+0000","last_became_peered":"2023-10-06T04:32:24.270394+0000","last_unstale":"2023-10-06T04:32:35.315277+0000","last_undegraded":"2023-10-06T04:32:35.315277+0000","last_fullsized":"2023-10-06T04:32:35.315277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269283+0000","last_change":"2023-10-06T04:32:23.452178+0000","last_active":"2023-10-06T04:32:34.269283+0000","last_peered":"2023-10-06T04:32:34.269283+0000","last_clean":"2023-10-06T04:32:34.269283+0000","last_became_active":"2023-10-06T04:32:23.451849+0000","last_became_peered":"2023-10-06T04:32:23.451849+0000","last_unstale":"2023-10-06T04:32:34.269283+0000","last_undegraded":"2023-10-06T04:32:34.269283+0000","last_fullsized":"2023-10-06T04:32:34.269283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.026875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269679+0000","last_change":"2023-10-06T04:32:24.265320+0000","last_active":"2023-10-06T04:32:34.269679+0000","last_peered":"2023-10-06T04:32:34.269679+0000","last_clean":"2023-10-06T04:32:34.269679+0000","last_became_active":"2023-10-06T04:32:24.264894+0000","last_became_peered":"2023-10-06T04:32:24.264894+0000","last_unstale":"2023-10-06T04:32:34.269679+0000","last_undegraded":"2023-10-06T04:32:34.269679+0000","last_fullsized":"2023-10-06T04:32:34.269679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262902+0000","last_change":"2023-10-06T04:32:23.444136+0000","last_active":"2023-10-06T04:32:34.262902+0000","last_peered":"2023-10-06T04:32:34.262902+0000","last_clean":"2023-10-06T04:32:34.262902+0000","last_became_active":"2023-10-06T04:32:23.443839+0000","last_became_peered":"2023-10-06T04:32:23.443839+0000","last_unstale":"2023-10-06T04:32:34.262902+0000","last_undegraded":"2023-10-06T04:32:34.262902+0000","last_fullsized":"2023-10-06T04:32:34.262902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:10.814479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268848+0000","last_change":"2023-10-06T04:32:24.269672+0000","last_active":"2023-10-06T04:32:34.268848+0000","last_peered":"2023-10-06T04:32:34.268848+0000","last_clean":"2023-10-06T04:32:34.268848+0000","last_became_active":"2023-10-06T04:32:24.269407+0000","last_became_peered":"2023-10-06T04:32:24.269407+0000","last_unstale":"2023-10-06T04:32:34.268848+0000","last_undegraded":"2023-10-06T04:32:34.268848+0000","last_fullsized":"2023-10-06T04:32:34.268848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269617+0000","last_change":"2023-10-06T04:32:23.446685+0000","last_active":"2023-10-06T04:32:34.269617+0000","last_peered":"2023-10-06T04:32:34.269617+0000","last_clean":"2023-10-06T04:32:34.269617+0000","last_became_active":"2023-10-06T04:32:23.446432+0000","last_became_peered":"2023-10-06T04:32:23.446432+0000","last_unstale":"2023-10-06T04:32:34.269617+0000","last_undegraded":"2023-10-06T04:32:34.269617+0000","last_fullsized":"2023-10-06T04:32:34.269617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268778+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:32:34.268778+0000","last_peered":"2023-10-06T04:32:34.268778+0000","last_clean":"2023-10-06T04:32:34.268778+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:32:34.268778+0000","last_undegraded":"2023-10-06T04:32:34.268778+0000","last_fullsized":"2023-10-06T04:32:34.268778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270090+0000","last_change":"2023-10-06T04:32:24.266084+0000","last_active":"2023-10-06T04:32:34.270090+0000","last_peered":"2023-10-06T04:32:34.270090+0000","last_clean":"2023-10-06T04:32:34.270090+0000","last_became_active":"2023-10-06T04:32:24.265847+0000","last_became_peered":"2023-10-06T04:32:24.265847+0000","last_unstale":"2023-10-06T04:32:34.270090+0000","last_undegraded":"2023-10-06T04:32:34.270090+0000","last_fullsized":"2023-10-06T04:32:34.270090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270247+0000","last_change":"2023-10-06T04:32:23.451106+0000","last_active":"2023-10-06T04:32:34.270247+0000","last_peered":"2023-10-06T04:32:34.270247+0000","last_clean":"2023-10-06T04:32:34.270247+0000","last_became_active":"2023-10-06T04:32:23.450838+0000","last_became_peered":"2023-10-06T04:32:23.450838+0000","last_unstale":"2023-10-06T04:32:34.270247+0000","last_undegraded":"2023-10-06T04:32:34.270247+0000","last_fullsized":"2023-10-06T04:32:34.270247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315391+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:32:35.315391+0000","last_peered":"2023-10-06T04:32:35.315391+0000","last_clean":"2023-10-06T04:32:35.315391+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:32:35.315391+0000","last_undegraded":"2023-10-06T04:32:35.315391+0000","last_fullsized":"2023-10-06T04:32:35.315391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269578+0000","last_change":"2023-10-06T04:32:24.272289+0000","last_active":"2023-10-06T04:32:34.269578+0000","last_peered":"2023-10-06T04:32:34.269578+0000","last_clean":"2023-10-06T04:32:34.269578+0000","last_became_active":"2023-10-06T04:32:24.271996+0000","last_became_peered":"2023-10-06T04:32:24.271996+0000","last_unstale":"2023-10-06T04:32:34.269578+0000","last_undegraded":"2023-10-06T04:32:34.269578+0000","last_fullsized":"2023-10-06T04:32:34.269578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'3","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263158+0000","last_change":"2023-10-06T04:32:23.444286+0000","last_active":"2023-10-06T04:32:34.263158+0000","last_peered":"2023-10-06T04:32:34.263158+0000","last_clean":"2023-10-06T04:32:34.263158+0000","last_became_active":"2023-10-06T04:32:23.443982+0000","last_became_peered":"2023-10-06T04:32:23.443982+0000","last_unstale":"2023-10-06T04:32:34.263158+0000","last_undegraded":"2023-10-06T04:32:34.263158+0000","last_fullsized":"2023-10-06T04:32:34.263158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:20.394340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262988+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:32:34.262988+0000","last_peered":"2023-10-06T04:32:34.262988+0000","last_clean":"2023-10-06T04:32:34.262988+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:32:34.262988+0000","last_undegraded":"2023-10-06T04:32:34.262988+0000","last_fullsized":"2023-10-06T04:32:34.262988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"34'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315767+0000","last_change":"2023-10-06T04:32:24.265180+0000","last_active":"2023-10-06T04:32:35.315767+0000","last_peered":"2023-10-06T04:32:35.315767+0000","last_clean":"2023-10-06T04:32:35.315767+0000","last_became_active":"2023-10-06T04:32:24.264769+0000","last_became_peered":"2023-10-06T04:32:24.264769+0000","last_unstale":"2023-10-06T04:32:35.315767+0000","last_undegraded":"2023-10-06T04:32:35.315767+0000","last_fullsized":"2023-10-06T04:32:35.315767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268997+0000","last_change":"2023-10-06T04:32:23.452281+0000","last_active":"2023-10-06T04:32:34.268997+0000","last_peered":"2023-10-06T04:32:34.268997+0000","last_clean":"2023-10-06T04:32:34.268997+0000","last_became_active":"2023-10-06T04:32:23.452023+0000","last_became_peered":"2023-10-06T04:32:23.452023+0000","last_unstale":"2023-10-06T04:32:34.268997+0000","last_undegraded":"2023-10-06T04:32:34.268997+0000","last_fullsized":"2023-10-06T04:32:34.268997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315453+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:32:35.315453+0000","last_peered":"2023-10-06T04:32:35.315453+0000","last_clean":"2023-10-06T04:32:35.315453+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:32:35.315453+0000","last_undegraded":"2023-10-06T04:32:35.315453+0000","last_fullsized":"2023-10-06T04:32:35.315453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269424+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:32:34.269424+0000","last_peered":"2023-10-06T04:32:34.269424+0000","last_clean":"2023-10-06T04:32:34.269424+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:32:34.269424+0000","last_undegraded":"2023-10-06T04:32:34.269424+0000","last_fullsized":"2023-10-06T04:32:34.269424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268400+0000","last_change":"2023-10-06T04:32:24.267047+0000","last_active":"2023-10-06T04:32:34.268400+0000","last_peered":"2023-10-06T04:32:34.268400+0000","last_clean":"2023-10-06T04:32:34.268400+0000","last_became_active":"2023-10-06T04:32:24.266777+0000","last_became_peered":"2023-10-06T04:32:24.266777+0000","last_unstale":"2023-10-06T04:32:34.268400+0000","last_undegraded":"2023-10-06T04:32:34.268400+0000","last_fullsized":"2023-10-06T04:32:34.268400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315859+0000","last_change":"2023-10-06T04:32:23.448480+0000","last_active":"2023-10-06T04:32:35.315859+0000","last_peered":"2023-10-06T04:32:35.315859+0000","last_clean":"2023-10-06T04:32:35.315859+0000","last_became_active":"2023-10-06T04:32:23.448242+0000","last_became_peered":"2023-10-06T04:32:23.448242+0000","last_unstale":"2023-10-06T04:32:35.315859+0000","last_undegraded":"2023-10-06T04:32:35.315859+0000","last_fullsized":"2023-10-06T04:32:35.315859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269418+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:32:34.269418+0000","last_peered":"2023-10-06T04:32:34.269418+0000","last_clean":"2023-10-06T04:32:34.269418+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:32:34.269418+0000","last_undegraded":"2023-10-06T04:32:34.269418+0000","last_fullsized":"2023-10-06T04:32:34.269418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268375+0000","last_change":"2023-10-06T04:32:24.266050+0000","last_active":"2023-10-06T04:32:34.268375+0000","last_peered":"2023-10-06T04:32:34.268375+0000","last_clean":"2023-10-06T04:32:34.268375+0000","last_became_active":"2023-10-06T04:32:24.265769+0000","last_became_peered":"2023-10-06T04:32:24.265769+0000","last_unstale":"2023-10-06T04:32:34.268375+0000","last_undegraded":"2023-10-06T04:32:34.268375+0000","last_fullsized":"2023-10-06T04:32:34.268375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:01.427293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268726+0000","last_change":"2023-10-06T04:32:23.443114+0000","last_active":"2023-10-06T04:32:34.268726+0000","last_peered":"2023-10-06T04:32:34.268726+0000","last_clean":"2023-10-06T04:32:34.268726+0000","last_became_active":"2023-10-06T04:32:23.442844+0000","last_became_peered":"2023-10-06T04:32:23.442844+0000","last_unstale":"2023-10-06T04:32:34.268726+0000","last_undegraded":"2023-10-06T04:32:34.268726+0000","last_fullsized":"2023-10-06T04:32:34.268726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269667+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:32:34.269667+0000","last_peered":"2023-10-06T04:32:34.269667+0000","last_clean":"2023-10-06T04:32:34.269667+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:32:34.269667+0000","last_undegraded":"2023-10-06T04:32:34.269667+0000","last_fullsized":"2023-10-06T04:32:34.269667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261395+0000","last_change":"2023-10-06T04:32:24.265505+0000","last_active":"2023-10-06T04:32:34.261395+0000","last_peered":"2023-10-06T04:32:34.261395+0000","last_clean":"2023-10-06T04:32:34.261395+0000","last_became_active":"2023-10-06T04:32:24.265228+0000","last_became_peered":"2023-10-06T04:32:24.265228+0000","last_unstale":"2023-10-06T04:32:34.261395+0000","last_undegraded":"2023-10-06T04:32:34.261395+0000","last_fullsized":"2023-10-06T04:32:34.261395+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260966+0000","last_change":"2023-10-06T04:32:23.438229+0000","last_active":"2023-10-06T04:32:34.260966+0000","last_peered":"2023-10-06T04:32:34.260966+0000","last_clean":"2023-10-06T04:32:34.260966+0000","last_became_active":"2023-10-06T04:32:23.437954+0000","last_became_peered":"2023-10-06T04:32:23.437954+0000","last_unstale":"2023-10-06T04:32:34.260966+0000","last_undegraded":"2023-10-06T04:32:34.260966+0000","last_fullsized":"2023-10-06T04:32:34.260966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:22.972008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268724+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:32:34.268724+0000","last_peered":"2023-10-06T04:32:34.268724+0000","last_clean":"2023-10-06T04:32:34.268724+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:32:34.268724+0000","last_undegraded":"2023-10-06T04:32:34.268724+0000","last_fullsized":"2023-10-06T04:32:34.268724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269083+0000","last_change":"2023-10-06T04:32:24.272223+0000","last_active":"2023-10-06T04:32:34.269083+0000","last_peered":"2023-10-06T04:32:34.269083+0000","last_clean":"2023-10-06T04:32:34.269083+0000","last_became_active":"2023-10-06T04:32:24.271905+0000","last_became_peered":"2023-10-06T04:32:24.271905+0000","last_unstale":"2023-10-06T04:32:34.269083+0000","last_undegraded":"2023-10-06T04:32:34.269083+0000","last_fullsized":"2023-10-06T04:32:34.269083+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:17.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269275+0000","last_change":"2023-10-06T04:32:23.444321+0000","last_active":"2023-10-06T04:32:34.269275+0000","last_peered":"2023-10-06T04:32:34.269275+0000","last_clean":"2023-10-06T04:32:34.269275+0000","last_became_active":"2023-10-06T04:32:23.444073+0000","last_became_peered":"2023-10-06T04:32:23.444073+0000","last_unstale":"2023-10-06T04:32:34.269275+0000","last_undegraded":"2023-10-06T04:32:34.269275+0000","last_fullsized":"2023-10-06T04:32:34.269275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:37.675115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269325+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:32:34.269325+0000","last_peered":"2023-10-06T04:32:34.269325+0000","last_clean":"2023-10-06T04:32:34.269325+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:32:34.269325+0000","last_undegraded":"2023-10-06T04:32:34.269325+0000","last_fullsized":"2023-10-06T04:32:34.269325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300340,"kb_used_data":3052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071500,"statfs":{"total":96636764160,"available":96329216000,"internally_reserved":0,"allocated":3125248,"data_stored":2869386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":188202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.496},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.81100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297700,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074140,"statfs":{"total":96636764160,"available":96331919360,"internally_reserved":0,"allocated":421888,"data_stored":175530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.88600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073536,"statfs":{"total":96636764160,"available":96331300864,"internally_reserved":0,"allocated":1040384,"data_stored":778570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.26100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.27200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297712,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074128,"statfs":{"total":96636764160,"available":96331907072,"internally_reserved":0,"allocated":434176,"data_stored":181866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.23100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.371}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301848,"kb_used_data":4532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069992,"statfs":{"total":96636764160,"available":96327671808,"internally_reserved":0,"allocated":4640768,"data_stored":4382506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.248},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.29999999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":188202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.495}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36299999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303844,"kb_used_data":6556,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067996,"statfs":{"total":96636764160,"available":96325627904,"internally_reserved":0,"allocated":6713344,"data_stored":6466986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.626}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:32:42.043 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:32:42.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.190+0000 7f6cef221700 1 Processor -- start 2023-10-06T04:32:42.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.190+0000 7f6cef221700 1 -- start start 2023-10-06T04:32:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.190+0000 7f6cef221700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce80ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.190+0000 7f6cef221700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ce8157330 0x7f6ce80adc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.190+0000 7f6cef221700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ce8157910 con 0x7f6ce8156f60 2023-10-06T04:32:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6cef221700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ce806d240 con 0x7f6ce8157330 2023-10-06T04:32:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6cef221700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce806d520 con 0x7f6ce806cd60 2023-10-06T04:32:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ce8157330 0x7f6ce80adc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f6ce8157330 0x7f6ce80adc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33610/0 (socket says 172.21.15.123:33610) 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 -- 172.21.15.123:0/1064597067 learned_addr learned my addr 172.21.15.123:0/1064597067 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ce7fff700 1 --2- 172.21.15.123:0/1064597067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce80ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 -- 172.21.15.123:0/1064597067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 msgr2=0x7f6ce80ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 --2- 172.21.15.123:0/1064597067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce80ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 -- 172.21.15.123:0/1064597067 >> v1:172.21.15.177:6789/0 conn(0x7f6ce8156f60 legacy=0x7f6ce815ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 -- 172.21.15.123:0/1064597067 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce806d800 con 0x7f6ce8157330 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ced7be700 1 --2- 172.21.15.123:0/1064597067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80adc70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009f20 tx=0x7f6ce0009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35815878a7a9f683 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ce77fe700 1 -- 172.21.15.123:0/1064597067 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce000e070 con 0x7f6ce8157330 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ce77fe700 1 -- 172.21.15.123:0/1064597067 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce000b9d0 con 0x7f6ce8157330 2023-10-06T04:32:42.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.191+0000 7f6ce77fe700 1 -- 172.21.15.123:0/1064597067 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce00092e0 con 0x7f6ce8157330 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 -- 172.21.15.123:0/1064597067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 msgr2=0x7f6ce80adc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 --2- 172.21.15.123:0/1064597067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80adc70 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7f6ce0009f20 tx=0x7f6ce0009840 comp rx=0 tx=0).stop 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 -- 172.21.15.123:0/1064597067 shutdown_connections 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 --2- 172.21.15.123:0/1064597067 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80adc70 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 --2- 172.21.15.123:0/1064597067 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce80ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 -- 172.21.15.123:0/1064597067 >> 172.21.15.123:0/1064597067 conn(0x7f6ce80ce0c0 msgr2=0x7f6ce80ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 -- 172.21.15.123:0/1064597067 shutdown_connections 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.192+0000 7f6cef221700 1 -- 172.21.15.123:0/1064597067 wait complete. 2023-10-06T04:32:42.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.193+0000 7f6cef221700 1 Processor -- start 2023-10-06T04:32:42.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.193+0000 7f6cef221700 1 -- start start 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6cef221700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce816a520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6cef221700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8156f60 0x7f6ce8153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6cef221700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80af680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6cef221700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce81541b0 con 0x7f6ce8157330 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6cef221700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce8155000 con 0x7f6ce8156f60 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6cef221700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ce81546d0 con 0x7f6ce806cd60 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80af680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80af680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33616/0 (socket says 172.21.15.123:33616) 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 -- 172.21.15.123:0/3283900050 learned_addr learned my addr 172.21.15.123:0/3283900050 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:42.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 -- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 msgr2=0x7f6ce816a520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6cecfbd700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce816a520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ce7fff700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8156f60 0x7f6ce8153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce816a520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 -- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8156f60 msgr2=0x7f6ce8153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8156f60 0x7f6ce8153980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.201 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.194+0000 7f6ced7be700 1 -- 172.21.15.123:0/3283900050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce0009ae0 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.195+0000 7f6ced7be700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80af680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000b610 tx=0x7f6ce0018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.195+0000 7f6ce4ff9700 1 -- 172.21.15.123:0/3283900050 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0008a40 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.195+0000 7f6ce4ff9700 1 -- 172.21.15.123:0/3283900050 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce0023070 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.195+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ce81549b0 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.195+0000 7f6ce4ff9700 1 -- 172.21.15.123:0/3283900050 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce0008dd0 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.195+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ce8153ec0 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.196+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ce8155170 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.196+0000 7f6ce4ff9700 1 -- 172.21.15.123:0/3283900050 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f6ce0025030 con 0x7f6ce8157330 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.197+0000 7f6ce4ff9700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6ccc066f30 0x7f6ccc0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.197+0000 7f6cecfbd700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6ccc066f30 0x7f6ccc0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.197+0000 7f6ce4ff9700 1 -- 172.21.15.123:0/3283900050 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f6ce008a3f0 con 0x7f6ce8157330 2023-10-06T04:32:42.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.197+0000 7f6cecfbd700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6ccc066f30 0x7f6ccc0693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80b0210 tx=0x7f6cdc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.201+0000 7f6ce4ff9700 1 -- 172.21.15.123:0/3283900050 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f6ce00572e0 con 0x7f6ce8157330 2023-10-06T04:32:42.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.378+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6ce806dae0 con 0x7f6ccc066f30 2023-10-06T04:32:42.390 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.388+0000 7f6ce4ff9700 1 -- 172.21.15.123:0/3283900050 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+386486 (secure 0 0 0) 0x7f6ce806dae0 con 0x7f6ccc066f30 2023-10-06T04:32:42.395 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6ccc066f30 msgr2=0x7f6ccc0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6ccc066f30 0x7f6ccc0693e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f6ce80b0210 tx=0x7f6cdc009040 comp rx=0 tx=0).stop 2023-10-06T04:32:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 msgr2=0x7f6ce80af680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80af680 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f6ce000b610 tx=0x7f6ce0018000 comp rx=0 tx=0).stop 2023-10-06T04:32:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 shutdown_connections 2023-10-06T04:32:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f6ccc066f30 0x7f6ccc0693e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f6ce8157330 0x7f6ce80af680 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f6ce806cd60 0x7f6ce816a520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 --2- 172.21.15.123:0/3283900050 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f6ce8156f60 0x7f6ce8153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 >> 172.21.15.123:0/3283900050 conn(0x7f6ce80ce0c0 msgr2=0x7f6ce80b1480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:42.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 shutdown_connections 2023-10-06T04:32:42.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.392+0000 7f6cef221700 1 -- 172.21.15.123:0/3283900050 wait complete. 2023-10-06T04:32:42.399 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:32:42.406 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":75,"stamp":"2023-10-06T04:32:40.902595+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":210,"acting":210,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395196,"kb_used_data":16864,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752579524,"statfs":{"total":773094113280,"available":770641432576,"internally_reserved":0,"allocated":17268736,"data_stored":15231248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":6289150,"num_objects":-20,"num_object_clones":0,"num_object_copies":-40,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-20,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":51,"num_write_kb":6167,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001036"},"pg_stats":[{"pgid":"6.3f","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270288+0000","last_change":"2023-10-06T04:32:24.268479+0000","last_active":"2023-10-06T04:32:34.270288+0000","last_peered":"2023-10-06T04:32:34.270288+0000","last_clean":"2023-10-06T04:32:34.270288+0000","last_became_active":"2023-10-06T04:32:24.268175+0000","last_became_peered":"2023-10-06T04:32:24.268175+0000","last_unstale":"2023-10-06T04:32:34.270288+0000","last_undegraded":"2023-10-06T04:32:34.270288+0000","last_fullsized":"2023-10-06T04:32:34.270288+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"34'4","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270143+0000","last_change":"2023-10-06T04:32:24.269331+0000","last_active":"2023-10-06T04:32:34.270143+0000","last_peered":"2023-10-06T04:32:34.270143+0000","last_clean":"2023-10-06T04:32:34.270143+0000","last_became_active":"2023-10-06T04:32:24.269096+0000","last_became_peered":"2023-10-06T04:32:24.269096+0000","last_unstale":"2023-10-06T04:32:34.270143+0000","last_undegraded":"2023-10-06T04:32:34.270143+0000","last_fullsized":"2023-10-06T04:32:34.270143+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270084+0000","last_change":"2023-10-06T04:32:24.267252+0000","last_active":"2023-10-06T04:32:34.270084+0000","last_peered":"2023-10-06T04:32:34.270084+0000","last_clean":"2023-10-06T04:32:34.270084+0000","last_became_active":"2023-10-06T04:32:24.266928+0000","last_became_peered":"2023-10-06T04:32:24.266928+0000","last_unstale":"2023-10-06T04:32:34.270084+0000","last_undegraded":"2023-10-06T04:32:34.270084+0000","last_fullsized":"2023-10-06T04:32:34.270084+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263072+0000","last_change":"2023-10-06T04:32:24.271555+0000","last_active":"2023-10-06T04:32:34.263072+0000","last_peered":"2023-10-06T04:32:34.263072+0000","last_clean":"2023-10-06T04:32:34.263072+0000","last_became_active":"2023-10-06T04:32:24.271271+0000","last_became_peered":"2023-10-06T04:32:24.271271+0000","last_unstale":"2023-10-06T04:32:34.263072+0000","last_undegraded":"2023-10-06T04:32:34.263072+0000","last_fullsized":"2023-10-06T04:32:34.263072+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"34'4","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263100+0000","last_change":"2023-10-06T04:32:24.270442+0000","last_active":"2023-10-06T04:32:34.263100+0000","last_peered":"2023-10-06T04:32:34.263100+0000","last_clean":"2023-10-06T04:32:34.263100+0000","last_became_active":"2023-10-06T04:32:24.270131+0000","last_became_peered":"2023-10-06T04:32:24.270131+0000","last_unstale":"2023-10-06T04:32:34.263100+0000","last_undegraded":"2023-10-06T04:32:34.263100+0000","last_fullsized":"2023-10-06T04:32:34.263100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268374+0000","last_change":"2023-10-06T04:32:24.263272+0000","last_active":"2023-10-06T04:32:34.268374+0000","last_peered":"2023-10-06T04:32:34.268374+0000","last_clean":"2023-10-06T04:32:34.268374+0000","last_became_active":"2023-10-06T04:32:24.262984+0000","last_became_peered":"2023-10-06T04:32:24.262984+0000","last_unstale":"2023-10-06T04:32:34.268374+0000","last_undegraded":"2023-10-06T04:32:34.268374+0000","last_fullsized":"2023-10-06T04:32:34.268374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:48.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315314+0000","last_change":"2023-10-06T04:32:24.263913+0000","last_active":"2023-10-06T04:32:35.315314+0000","last_peered":"2023-10-06T04:32:35.315314+0000","last_clean":"2023-10-06T04:32:35.315314+0000","last_became_active":"2023-10-06T04:32:24.263358+0000","last_became_peered":"2023-10-06T04:32:24.263358+0000","last_unstale":"2023-10-06T04:32:35.315314+0000","last_undegraded":"2023-10-06T04:32:35.315314+0000","last_fullsized":"2023-10-06T04:32:35.315314+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269238+0000","last_change":"2023-10-06T04:32:24.266865+0000","last_active":"2023-10-06T04:32:34.269238+0000","last_peered":"2023-10-06T04:32:34.269238+0000","last_clean":"2023-10-06T04:32:34.269238+0000","last_became_active":"2023-10-06T04:32:24.266456+0000","last_became_peered":"2023-10-06T04:32:24.266456+0000","last_unstale":"2023-10-06T04:32:34.269238+0000","last_undegraded":"2023-10-06T04:32:34.269238+0000","last_fullsized":"2023-10-06T04:32:34.269238+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"34'11","reported_seq":23,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270525+0000","last_change":"2023-10-06T04:32:24.267567+0000","last_active":"2023-10-06T04:32:34.270525+0000","last_peered":"2023-10-06T04:32:34.270525+0000","last_clean":"2023-10-06T04:32:34.270525+0000","last_became_active":"2023-10-06T04:32:24.267326+0000","last_became_peered":"2023-10-06T04:32:24.267326+0000","last_unstale":"2023-10-06T04:32:34.270525+0000","last_undegraded":"2023-10-06T04:32:34.270525+0000","last_fullsized":"2023-10-06T04:32:34.270525+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"34'15","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315604+0000","last_change":"2023-10-06T04:32:24.266439+0000","last_active":"2023-10-06T04:32:35.315604+0000","last_peered":"2023-10-06T04:32:35.315604+0000","last_clean":"2023-10-06T04:32:35.315604+0000","last_became_active":"2023-10-06T04:32:24.266172+0000","last_became_peered":"2023-10-06T04:32:24.266172+0000","last_unstale":"2023-10-06T04:32:35.315604+0000","last_undegraded":"2023-10-06T04:32:35.315604+0000","last_fullsized":"2023-10-06T04:32:35.315604+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:07.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"34'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315594+0000","last_change":"2023-10-06T04:32:24.264976+0000","last_active":"2023-10-06T04:32:35.315594+0000","last_peered":"2023-10-06T04:32:35.315594+0000","last_clean":"2023-10-06T04:32:35.315594+0000","last_became_active":"2023-10-06T04:32:24.264647+0000","last_became_peered":"2023-10-06T04:32:24.264647+0000","last_unstale":"2023-10-06T04:32:35.315594+0000","last_undegraded":"2023-10-06T04:32:35.315594+0000","last_fullsized":"2023-10-06T04:32:35.315594+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269897+0000","last_change":"2023-10-06T04:32:24.262121+0000","last_active":"2023-10-06T04:32:34.269897+0000","last_peered":"2023-10-06T04:32:34.269897+0000","last_clean":"2023-10-06T04:32:34.269897+0000","last_became_active":"2023-10-06T04:32:24.261846+0000","last_became_peered":"2023-10-06T04:32:24.261846+0000","last_unstale":"2023-10-06T04:32:34.269897+0000","last_undegraded":"2023-10-06T04:32:34.269897+0000","last_fullsized":"2023-10-06T04:32:34.269897+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315289+0000","last_change":"2023-10-06T04:32:24.267116+0000","last_active":"2023-10-06T04:32:35.315289+0000","last_peered":"2023-10-06T04:32:35.315289+0000","last_clean":"2023-10-06T04:32:35.315289+0000","last_became_active":"2023-10-06T04:32:24.266756+0000","last_became_peered":"2023-10-06T04:32:24.266756+0000","last_unstale":"2023-10-06T04:32:35.315289+0000","last_undegraded":"2023-10-06T04:32:35.315289+0000","last_fullsized":"2023-10-06T04:32:35.315289+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:26.457033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261300+0000","last_change":"2023-10-06T04:32:24.267554+0000","last_active":"2023-10-06T04:32:34.261300+0000","last_peered":"2023-10-06T04:32:34.261300+0000","last_clean":"2023-10-06T04:32:34.261300+0000","last_became_active":"2023-10-06T04:32:24.266198+0000","last_became_peered":"2023-10-06T04:32:24.266198+0000","last_unstale":"2023-10-06T04:32:34.261300+0000","last_undegraded":"2023-10-06T04:32:34.261300+0000","last_fullsized":"2023-10-06T04:32:34.261300+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315261+0000","last_change":"2023-10-06T04:32:24.265234+0000","last_active":"2023-10-06T04:32:35.315261+0000","last_peered":"2023-10-06T04:32:35.315261+0000","last_clean":"2023-10-06T04:32:35.315261+0000","last_became_active":"2023-10-06T04:32:24.264892+0000","last_became_peered":"2023-10-06T04:32:24.264892+0000","last_unstale":"2023-10-06T04:32:35.315261+0000","last_undegraded":"2023-10-06T04:32:35.315261+0000","last_fullsized":"2023-10-06T04:32:35.315261+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268527+0000","last_change":"2023-10-06T04:32:24.268696+0000","last_active":"2023-10-06T04:32:34.268527+0000","last_peered":"2023-10-06T04:32:34.268527+0000","last_clean":"2023-10-06T04:32:34.268527+0000","last_became_active":"2023-10-06T04:32:24.268430+0000","last_became_peered":"2023-10-06T04:32:24.268430+0000","last_unstale":"2023-10-06T04:32:34.268527+0000","last_undegraded":"2023-10-06T04:32:34.268527+0000","last_fullsized":"2023-10-06T04:32:34.268527+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268528+0000","last_change":"2023-10-06T04:32:24.269272+0000","last_active":"2023-10-06T04:32:34.268528+0000","last_peered":"2023-10-06T04:32:34.268528+0000","last_clean":"2023-10-06T04:32:34.268528+0000","last_became_active":"2023-10-06T04:32:24.268793+0000","last_became_peered":"2023-10-06T04:32:24.268793+0000","last_unstale":"2023-10-06T04:32:34.268528+0000","last_undegraded":"2023-10-06T04:32:34.268528+0000","last_fullsized":"2023-10-06T04:32:34.268528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"34'10","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268890+0000","last_change":"2023-10-06T04:32:24.268089+0000","last_active":"2023-10-06T04:32:34.268890+0000","last_peered":"2023-10-06T04:32:34.268890+0000","last_clean":"2023-10-06T04:32:34.268890+0000","last_became_active":"2023-10-06T04:32:24.267842+0000","last_became_peered":"2023-10-06T04:32:24.267842+0000","last_unstale":"2023-10-06T04:32:34.268890+0000","last_undegraded":"2023-10-06T04:32:34.268890+0000","last_fullsized":"2023-10-06T04:32:34.268890+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:36.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"34'7","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263103+0000","last_change":"2023-10-06T04:32:24.263185+0000","last_active":"2023-10-06T04:32:34.263103+0000","last_peered":"2023-10-06T04:32:34.263103+0000","last_clean":"2023-10-06T04:32:34.263103+0000","last_became_active":"2023-10-06T04:32:24.262500+0000","last_became_peered":"2023-10-06T04:32:24.262500+0000","last_unstale":"2023-10-06T04:32:34.263103+0000","last_undegraded":"2023-10-06T04:32:34.263103+0000","last_fullsized":"2023-10-06T04:32:34.263103+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:18.096304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268374+0000","last_change":"2023-10-06T04:32:24.266364+0000","last_active":"2023-10-06T04:32:34.268374+0000","last_peered":"2023-10-06T04:32:34.268374+0000","last_clean":"2023-10-06T04:32:34.268374+0000","last_became_active":"2023-10-06T04:32:24.266082+0000","last_became_peered":"2023-10-06T04:32:24.266082+0000","last_unstale":"2023-10-06T04:32:34.268374+0000","last_undegraded":"2023-10-06T04:32:34.268374+0000","last_fullsized":"2023-10-06T04:32:34.268374+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"34'11","reported_seq":24,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315320+0000","last_change":"2023-10-06T04:32:24.265143+0000","last_active":"2023-10-06T04:32:35.315320+0000","last_peered":"2023-10-06T04:32:35.315320+0000","last_clean":"2023-10-06T04:32:35.315320+0000","last_became_active":"2023-10-06T04:32:24.264481+0000","last_became_peered":"2023-10-06T04:32:24.264481+0000","last_unstale":"2023-10-06T04:32:35.315320+0000","last_undegraded":"2023-10-06T04:32:35.315320+0000","last_fullsized":"2023-10-06T04:32:35.315320+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:04.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260770+0000","last_change":"2023-10-06T04:32:24.262815+0000","last_active":"2023-10-06T04:32:34.260770+0000","last_peered":"2023-10-06T04:32:34.260770+0000","last_clean":"2023-10-06T04:32:34.260770+0000","last_became_active":"2023-10-06T04:32:24.262576+0000","last_became_peered":"2023-10-06T04:32:24.262576+0000","last_unstale":"2023-10-06T04:32:34.260770+0000","last_undegraded":"2023-10-06T04:32:34.260770+0000","last_fullsized":"2023-10-06T04:32:34.260770+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"34'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269183+0000","last_change":"2023-10-06T04:32:24.269201+0000","last_active":"2023-10-06T04:32:34.269183+0000","last_peered":"2023-10-06T04:32:34.269183+0000","last_clean":"2023-10-06T04:32:34.269183+0000","last_became_active":"2023-10-06T04:32:24.268841+0000","last_became_peered":"2023-10-06T04:32:24.268841+0000","last_unstale":"2023-10-06T04:32:34.269183+0000","last_undegraded":"2023-10-06T04:32:34.269183+0000","last_fullsized":"2023-10-06T04:32:34.269183+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"34'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262874+0000","last_change":"2023-10-06T04:32:24.264875+0000","last_active":"2023-10-06T04:32:34.262874+0000","last_peered":"2023-10-06T04:32:34.262874+0000","last_clean":"2023-10-06T04:32:34.262874+0000","last_became_active":"2023-10-06T04:32:24.264601+0000","last_became_peered":"2023-10-06T04:32:24.264601+0000","last_unstale":"2023-10-06T04:32:34.262874+0000","last_undegraded":"2023-10-06T04:32:34.262874+0000","last_fullsized":"2023-10-06T04:32:34.262874+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:38.680988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"34'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269157+0000","last_change":"2023-10-06T04:32:24.268714+0000","last_active":"2023-10-06T04:32:34.269157+0000","last_peered":"2023-10-06T04:32:34.269157+0000","last_clean":"2023-10-06T04:32:34.269157+0000","last_became_active":"2023-10-06T04:32:24.268440+0000","last_became_peered":"2023-10-06T04:32:24.268440+0000","last_unstale":"2023-10-06T04:32:34.269157+0000","last_undegraded":"2023-10-06T04:32:34.269157+0000","last_fullsized":"2023-10-06T04:32:34.269157+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:24.913639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268884+0000","last_change":"2023-10-06T04:32:24.266945+0000","last_active":"2023-10-06T04:32:34.268884+0000","last_peered":"2023-10-06T04:32:34.268884+0000","last_clean":"2023-10-06T04:32:34.268884+0000","last_became_active":"2023-10-06T04:32:24.266673+0000","last_became_peered":"2023-10-06T04:32:24.266673+0000","last_unstale":"2023-10-06T04:32:34.268884+0000","last_undegraded":"2023-10-06T04:32:34.268884+0000","last_fullsized":"2023-10-06T04:32:34.268884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"34'6","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268528+0000","last_change":"2023-10-06T04:32:24.270531+0000","last_active":"2023-10-06T04:32:34.268528+0000","last_peered":"2023-10-06T04:32:34.268528+0000","last_clean":"2023-10-06T04:32:34.268528+0000","last_became_active":"2023-10-06T04:32:24.270268+0000","last_became_peered":"2023-10-06T04:32:24.270268+0000","last_unstale":"2023-10-06T04:32:34.268528+0000","last_undegraded":"2023-10-06T04:32:34.268528+0000","last_fullsized":"2023-10-06T04:32:34.268528+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:34.859497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"34'4","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315561+0000","last_change":"2023-10-06T04:32:24.265317+0000","last_active":"2023-10-06T04:32:35.315561+0000","last_peered":"2023-10-06T04:32:35.315561+0000","last_clean":"2023-10-06T04:32:35.315561+0000","last_became_active":"2023-10-06T04:32:24.264377+0000","last_became_peered":"2023-10-06T04:32:24.264377+0000","last_unstale":"2023-10-06T04:32:35.315561+0000","last_undegraded":"2023-10-06T04:32:35.315561+0000","last_fullsized":"2023-10-06T04:32:35.315561+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"34'7","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268365+0000","last_change":"2023-10-06T04:32:24.264442+0000","last_active":"2023-10-06T04:32:34.268365+0000","last_peered":"2023-10-06T04:32:34.268365+0000","last_clean":"2023-10-06T04:32:34.268365+0000","last_became_active":"2023-10-06T04:32:24.264148+0000","last_became_peered":"2023-10-06T04:32:24.264148+0000","last_unstale":"2023-10-06T04:32:34.268365+0000","last_undegraded":"2023-10-06T04:32:34.268365+0000","last_fullsized":"2023-10-06T04:32:34.268365+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"34'8","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268651+0000","last_change":"2023-10-06T04:32:24.270800+0000","last_active":"2023-10-06T04:32:34.268651+0000","last_peered":"2023-10-06T04:32:34.268651+0000","last_clean":"2023-10-06T04:32:34.268651+0000","last_became_active":"2023-10-06T04:32:24.270560+0000","last_became_peered":"2023-10-06T04:32:24.270560+0000","last_unstale":"2023-10-06T04:32:34.268651+0000","last_undegraded":"2023-10-06T04:32:34.268651+0000","last_fullsized":"2023-10-06T04:32:34.268651+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"34'12","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269073+0000","last_change":"2023-10-06T04:32:24.271609+0000","last_active":"2023-10-06T04:32:34.269073+0000","last_peered":"2023-10-06T04:32:34.269073+0000","last_clean":"2023-10-06T04:32:34.269073+0000","last_became_active":"2023-10-06T04:32:24.271354+0000","last_became_peered":"2023-10-06T04:32:24.271354+0000","last_unstale":"2023-10-06T04:32:34.269073+0000","last_undegraded":"2023-10-06T04:32:34.269073+0000","last_fullsized":"2023-10-06T04:32:34.269073+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270603+0000","last_change":"2023-10-06T04:32:24.263719+0000","last_active":"2023-10-06T04:32:34.270603+0000","last_peered":"2023-10-06T04:32:34.270603+0000","last_clean":"2023-10-06T04:32:34.270603+0000","last_became_active":"2023-10-06T04:32:24.263442+0000","last_became_peered":"2023-10-06T04:32:24.263442+0000","last_unstale":"2023-10-06T04:32:34.270603+0000","last_undegraded":"2023-10-06T04:32:34.270603+0000","last_fullsized":"2023-10-06T04:32:34.270603+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:40.811547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1c","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268733+0000","last_change":"2023-10-06T04:32:24.270093+0000","last_active":"2023-10-06T04:32:34.268733+0000","last_peered":"2023-10-06T04:32:34.268733+0000","last_clean":"2023-10-06T04:32:34.268733+0000","last_became_active":"2023-10-06T04:32:24.269851+0000","last_became_peered":"2023-10-06T04:32:24.269851+0000","last_unstale":"2023-10-06T04:32:34.268733+0000","last_undegraded":"2023-10-06T04:32:34.268733+0000","last_fullsized":"2023-10-06T04:32:34.268733+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'8","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269249+0000","last_change":"2023-10-06T04:32:23.441194+0000","last_active":"2023-10-06T04:32:34.269249+0000","last_peered":"2023-10-06T04:32:34.269249+0000","last_clean":"2023-10-06T04:32:34.269249+0000","last_became_active":"2023-10-06T04:32:23.440849+0000","last_became_peered":"2023-10-06T04:32:23.440849+0000","last_unstale":"2023-10-06T04:32:34.269249+0000","last_undegraded":"2023-10-06T04:32:34.269249+0000","last_fullsized":"2023-10-06T04:32:34.269249+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:59.829363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1d","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268727+0000","last_change":"2023-10-06T04:32:24.267279+0000","last_active":"2023-10-06T04:32:34.268727+0000","last_peered":"2023-10-06T04:32:34.268727+0000","last_clean":"2023-10-06T04:32:34.268727+0000","last_became_active":"2023-10-06T04:32:24.267016+0000","last_became_peered":"2023-10-06T04:32:24.267016+0000","last_unstale":"2023-10-06T04:32:34.268727+0000","last_undegraded":"2023-10-06T04:32:34.268727+0000","last_fullsized":"2023-10-06T04:32:34.268727+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268798+0000","last_change":"2023-10-06T04:32:23.450031+0000","last_active":"2023-10-06T04:32:34.268798+0000","last_peered":"2023-10-06T04:32:34.268798+0000","last_clean":"2023-10-06T04:32:34.268798+0000","last_became_active":"2023-10-06T04:32:23.449717+0000","last_became_peered":"2023-10-06T04:32:23.449717+0000","last_unstale":"2023-10-06T04:32:34.268798+0000","last_undegraded":"2023-10-06T04:32:34.268798+0000","last_fullsized":"2023-10-06T04:32:34.268798+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:19.596388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1e","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260810+0000","last_change":"2023-10-06T04:32:24.267302+0000","last_active":"2023-10-06T04:32:34.260810+0000","last_peered":"2023-10-06T04:32:34.260810+0000","last_clean":"2023-10-06T04:32:34.260810+0000","last_became_active":"2023-10-06T04:32:24.266976+0000","last_became_peered":"2023-10-06T04:32:24.266976+0000","last_unstale":"2023-10-06T04:32:34.260810+0000","last_undegraded":"2023-10-06T04:32:34.260810+0000","last_fullsized":"2023-10-06T04:32:34.260810+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268890+0000","last_change":"2023-10-06T04:32:23.447029+0000","last_active":"2023-10-06T04:32:34.268890+0000","last_peered":"2023-10-06T04:32:34.268890+0000","last_clean":"2023-10-06T04:32:34.268890+0000","last_became_active":"2023-10-06T04:32:23.446659+0000","last_became_peered":"2023-10-06T04:32:23.446659+0000","last_unstale":"2023-10-06T04:32:34.268890+0000","last_undegraded":"2023-10-06T04:32:34.268890+0000","last_fullsized":"2023-10-06T04:32:34.268890+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:13.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1f","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270340+0000","last_change":"2023-10-06T04:32:24.265927+0000","last_active":"2023-10-06T04:32:34.270340+0000","last_peered":"2023-10-06T04:32:34.270340+0000","last_clean":"2023-10-06T04:32:34.270340+0000","last_became_active":"2023-10-06T04:32:24.265668+0000","last_became_peered":"2023-10-06T04:32:24.265668+0000","last_unstale":"2023-10-06T04:32:34.270340+0000","last_undegraded":"2023-10-06T04:32:34.270340+0000","last_fullsized":"2023-10-06T04:32:34.270340+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260994+0000","last_change":"2023-10-06T04:32:23.437810+0000","last_active":"2023-10-06T04:32:34.260994+0000","last_peered":"2023-10-06T04:32:34.260994+0000","last_clean":"2023-10-06T04:32:34.260994+0000","last_became_active":"2023-10-06T04:32:23.437533+0000","last_became_peered":"2023-10-06T04:32:23.437533+0000","last_unstale":"2023-10-06T04:32:34.260994+0000","last_undegraded":"2023-10-06T04:32:34.260994+0000","last_fullsized":"2023-10-06T04:32:34.260994+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:46.149228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.18","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268666+0000","last_change":"2023-10-06T04:32:24.261934+0000","last_active":"2023-10-06T04:32:34.268666+0000","last_peered":"2023-10-06T04:32:34.268666+0000","last_clean":"2023-10-06T04:32:34.268666+0000","last_became_active":"2023-10-06T04:32:24.261681+0000","last_became_peered":"2023-10-06T04:32:24.261681+0000","last_unstale":"2023-10-06T04:32:34.268666+0000","last_undegraded":"2023-10-06T04:32:34.268666+0000","last_fullsized":"2023-10-06T04:32:34.268666+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315427+0000","last_change":"2023-10-06T04:32:23.443656+0000","last_active":"2023-10-06T04:32:35.315427+0000","last_peered":"2023-10-06T04:32:35.315427+0000","last_clean":"2023-10-06T04:32:35.315427+0000","last_became_active":"2023-10-06T04:32:23.443293+0000","last_became_peered":"2023-10-06T04:32:23.443293+0000","last_unstale":"2023-10-06T04:32:35.315427+0000","last_undegraded":"2023-10-06T04:32:35.315427+0000","last_fullsized":"2023-10-06T04:32:35.315427+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.19","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315387+0000","last_change":"2023-10-06T04:32:24.264104+0000","last_active":"2023-10-06T04:32:35.315387+0000","last_peered":"2023-10-06T04:32:35.315387+0000","last_clean":"2023-10-06T04:32:35.315387+0000","last_became_active":"2023-10-06T04:32:24.263651+0000","last_became_peered":"2023-10-06T04:32:24.263651+0000","last_unstale":"2023-10-06T04:32:35.315387+0000","last_undegraded":"2023-10-06T04:32:35.315387+0000","last_fullsized":"2023-10-06T04:32:35.315387+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:24.395312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269261+0000","last_change":"2023-10-06T04:32:23.451767+0000","last_active":"2023-10-06T04:32:34.269261+0000","last_peered":"2023-10-06T04:32:34.269261+0000","last_clean":"2023-10-06T04:32:34.269261+0000","last_became_active":"2023-10-06T04:32:23.451430+0000","last_became_peered":"2023-10-06T04:32:23.451430+0000","last_unstale":"2023-10-06T04:32:34.269261+0000","last_undegraded":"2023-10-06T04:32:34.269261+0000","last_fullsized":"2023-10-06T04:32:34.269261+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1a","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261339+0000","last_change":"2023-10-06T04:32:24.267750+0000","last_active":"2023-10-06T04:32:34.261339+0000","last_peered":"2023-10-06T04:32:34.261339+0000","last_clean":"2023-10-06T04:32:34.261339+0000","last_became_active":"2023-10-06T04:32:24.265805+0000","last_became_peered":"2023-10-06T04:32:24.265805+0000","last_unstale":"2023-10-06T04:32:34.261339+0000","last_undegraded":"2023-10-06T04:32:34.261339+0000","last_fullsized":"2023-10-06T04:32:34.261339+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268431+0000","last_change":"2023-10-06T04:32:23.449278+0000","last_active":"2023-10-06T04:32:34.268431+0000","last_peered":"2023-10-06T04:32:34.268431+0000","last_clean":"2023-10-06T04:32:34.268431+0000","last_became_active":"2023-10-06T04:32:23.449034+0000","last_became_peered":"2023-10-06T04:32:23.449034+0000","last_unstale":"2023-10-06T04:32:34.268431+0000","last_undegraded":"2023-10-06T04:32:34.268431+0000","last_fullsized":"2023-10-06T04:32:34.268431+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1b","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269734+0000","last_change":"2023-10-06T04:32:24.265155+0000","last_active":"2023-10-06T04:32:34.269734+0000","last_peered":"2023-10-06T04:32:34.269734+0000","last_clean":"2023-10-06T04:32:34.269734+0000","last_became_active":"2023-10-06T04:32:24.264863+0000","last_became_peered":"2023-10-06T04:32:24.264863+0000","last_unstale":"2023-10-06T04:32:34.269734+0000","last_undegraded":"2023-10-06T04:32:34.269734+0000","last_fullsized":"2023-10-06T04:32:34.269734+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260606+0000","last_change":"2023-10-06T04:32:23.440295+0000","last_active":"2023-10-06T04:32:34.260606+0000","last_peered":"2023-10-06T04:32:34.260606+0000","last_clean":"2023-10-06T04:32:34.260606+0000","last_became_active":"2023-10-06T04:32:23.440039+0000","last_became_peered":"2023-10-06T04:32:23.440039+0000","last_unstale":"2023-10-06T04:32:34.260606+0000","last_undegraded":"2023-10-06T04:32:34.260606+0000","last_fullsized":"2023-10-06T04:32:34.260606+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.14","version":"34'11","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263151+0000","last_change":"2023-10-06T04:32:24.271227+0000","last_active":"2023-10-06T04:32:34.263151+0000","last_peered":"2023-10-06T04:32:34.263151+0000","last_clean":"2023-10-06T04:32:34.263151+0000","last_became_active":"2023-10-06T04:32:24.270984+0000","last_became_peered":"2023-10-06T04:32:24.270984+0000","last_unstale":"2023-10-06T04:32:34.263151+0000","last_undegraded":"2023-10-06T04:32:34.263151+0000","last_fullsized":"2023-10-06T04:32:34.263151+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270397+0000","last_change":"2023-10-06T04:32:23.447791+0000","last_active":"2023-10-06T04:32:34.270397+0000","last_peered":"2023-10-06T04:32:34.270397+0000","last_clean":"2023-10-06T04:32:34.270397+0000","last_became_active":"2023-10-06T04:32:23.447395+0000","last_became_peered":"2023-10-06T04:32:23.447395+0000","last_unstale":"2023-10-06T04:32:34.270397+0000","last_undegraded":"2023-10-06T04:32:34.270397+0000","last_fullsized":"2023-10-06T04:32:34.270397+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:18.839316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.15","version":"34'6","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269473+0000","last_change":"2023-10-06T04:32:24.269565+0000","last_active":"2023-10-06T04:32:34.269473+0000","last_peered":"2023-10-06T04:32:34.269473+0000","last_clean":"2023-10-06T04:32:34.269473+0000","last_became_active":"2023-10-06T04:32:24.269285+0000","last_became_peered":"2023-10-06T04:32:24.269285+0000","last_unstale":"2023-10-06T04:32:34.269473+0000","last_undegraded":"2023-10-06T04:32:34.269473+0000","last_fullsized":"2023-10-06T04:32:34.269473+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315749+0000","last_change":"2023-10-06T04:32:23.444329+0000","last_active":"2023-10-06T04:32:35.315749+0000","last_peered":"2023-10-06T04:32:35.315749+0000","last_clean":"2023-10-06T04:32:35.315749+0000","last_became_active":"2023-10-06T04:32:23.444020+0000","last_became_peered":"2023-10-06T04:32:23.444020+0000","last_unstale":"2023-10-06T04:32:35.315749+0000","last_undegraded":"2023-10-06T04:32:35.315749+0000","last_fullsized":"2023-10-06T04:32:35.315749+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.16","version":"34'10","reported_seq":22,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268693+0000","last_change":"2023-10-06T04:32:24.264060+0000","last_active":"2023-10-06T04:32:34.268693+0000","last_peered":"2023-10-06T04:32:34.268693+0000","last_clean":"2023-10-06T04:32:34.268693+0000","last_became_active":"2023-10-06T04:32:24.263825+0000","last_became_peered":"2023-10-06T04:32:24.263825+0000","last_unstale":"2023-10-06T04:32:34.268693+0000","last_undegraded":"2023-10-06T04:32:34.268693+0000","last_fullsized":"2023-10-06T04:32:34.268693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'5","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315712+0000","last_change":"2023-10-06T04:32:23.447887+0000","last_active":"2023-10-06T04:32:35.315712+0000","last_peered":"2023-10-06T04:32:35.315712+0000","last_clean":"2023-10-06T04:32:35.315712+0000","last_became_active":"2023-10-06T04:32:23.447650+0000","last_became_peered":"2023-10-06T04:32:23.447650+0000","last_unstale":"2023-10-06T04:32:35.315712+0000","last_undegraded":"2023-10-06T04:32:35.315712+0000","last_fullsized":"2023-10-06T04:32:35.315712+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:47.164678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.17","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260931+0000","last_change":"2023-10-06T04:32:24.270050+0000","last_active":"2023-10-06T04:32:34.260931+0000","last_peered":"2023-10-06T04:32:34.260931+0000","last_clean":"2023-10-06T04:32:34.260931+0000","last_became_active":"2023-10-06T04:32:24.269298+0000","last_became_peered":"2023-10-06T04:32:24.269298+0000","last_unstale":"2023-10-06T04:32:34.260931+0000","last_undegraded":"2023-10-06T04:32:34.260931+0000","last_fullsized":"2023-10-06T04:32:34.260931+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'22","reported_seq":35,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269989+0000","last_change":"2023-10-06T04:32:23.447643+0000","last_active":"2023-10-06T04:32:34.269989+0000","last_peered":"2023-10-06T04:32:34.269989+0000","last_clean":"2023-10-06T04:32:34.269989+0000","last_became_active":"2023-10-06T04:32:23.447352+0000","last_became_peered":"2023-10-06T04:32:23.447352+0000","last_unstale":"2023-10-06T04:32:34.269989+0000","last_undegraded":"2023-10-06T04:32:34.269989+0000","last_fullsized":"2023-10-06T04:32:34.269989+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.10","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268957+0000","last_change":"2023-10-06T04:32:24.267086+0000","last_active":"2023-10-06T04:32:34.268957+0000","last_peered":"2023-10-06T04:32:34.268957+0000","last_clean":"2023-10-06T04:32:34.268957+0000","last_became_active":"2023-10-06T04:32:24.266829+0000","last_became_peered":"2023-10-06T04:32:24.266829+0000","last_unstale":"2023-10-06T04:32:34.268957+0000","last_undegraded":"2023-10-06T04:32:34.268957+0000","last_fullsized":"2023-10-06T04:32:34.268957+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'5","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269791+0000","last_change":"2023-10-06T04:32:23.443222+0000","last_active":"2023-10-06T04:32:34.269791+0000","last_peered":"2023-10-06T04:32:34.269791+0000","last_clean":"2023-10-06T04:32:34.269791+0000","last_became_active":"2023-10-06T04:32:23.442957+0000","last_became_peered":"2023-10-06T04:32:23.442957+0000","last_unstale":"2023-10-06T04:32:34.269791+0000","last_undegraded":"2023-10-06T04:32:34.269791+0000","last_fullsized":"2023-10-06T04:32:34.269791+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.11","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269309+0000","last_change":"2023-10-06T04:32:24.268660+0000","last_active":"2023-10-06T04:32:34.269309+0000","last_peered":"2023-10-06T04:32:34.269309+0000","last_clean":"2023-10-06T04:32:34.269309+0000","last_became_active":"2023-10-06T04:32:24.268397+0000","last_became_peered":"2023-10-06T04:32:24.268397+0000","last_unstale":"2023-10-06T04:32:34.269309+0000","last_undegraded":"2023-10-06T04:32:34.269309+0000","last_fullsized":"2023-10-06T04:32:34.269309+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268494+0000","last_change":"2023-10-06T04:32:23.445203+0000","last_active":"2023-10-06T04:32:34.268494+0000","last_peered":"2023-10-06T04:32:34.268494+0000","last_clean":"2023-10-06T04:32:34.268494+0000","last_became_active":"2023-10-06T04:32:23.444688+0000","last_became_peered":"2023-10-06T04:32:23.444688+0000","last_unstale":"2023-10-06T04:32:34.268494+0000","last_undegraded":"2023-10-06T04:32:34.268494+0000","last_fullsized":"2023-10-06T04:32:34.268494+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.12","version":"34'10","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269146+0000","last_change":"2023-10-06T04:32:24.271729+0000","last_active":"2023-10-06T04:32:34.269146+0000","last_peered":"2023-10-06T04:32:34.269146+0000","last_clean":"2023-10-06T04:32:34.269146+0000","last_became_active":"2023-10-06T04:32:24.271453+0000","last_became_peered":"2023-10-06T04:32:24.271453+0000","last_unstale":"2023-10-06T04:32:34.269146+0000","last_undegraded":"2023-10-06T04:32:34.269146+0000","last_fullsized":"2023-10-06T04:32:34.269146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268204+0000","last_change":"2023-10-06T04:32:23.440530+0000","last_active":"2023-10-06T04:32:34.268204+0000","last_peered":"2023-10-06T04:32:34.268204+0000","last_clean":"2023-10-06T04:32:34.268204+0000","last_became_active":"2023-10-06T04:32:23.440226+0000","last_became_peered":"2023-10-06T04:32:23.440226+0000","last_unstale":"2023-10-06T04:32:34.268204+0000","last_undegraded":"2023-10-06T04:32:34.268204+0000","last_fullsized":"2023-10-06T04:32:34.268204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.13","version":"34'15","reported_seq":27,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269621+0000","last_change":"2023-10-06T04:32:24.268363+0000","last_active":"2023-10-06T04:32:34.269621+0000","last_peered":"2023-10-06T04:32:34.269621+0000","last_clean":"2023-10-06T04:32:34.269621+0000","last_became_active":"2023-10-06T04:32:24.268116+0000","last_became_peered":"2023-10-06T04:32:24.268116+0000","last_unstale":"2023-10-06T04:32:34.269621+0000","last_undegraded":"2023-10-06T04:32:34.269621+0000","last_fullsized":"2023-10-06T04:32:34.269621+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'15","reported_seq":29,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268764+0000","last_change":"2023-10-06T04:32:23.452500+0000","last_active":"2023-10-06T04:32:34.268764+0000","last_peered":"2023-10-06T04:32:34.268764+0000","last_clean":"2023-10-06T04:32:34.268764+0000","last_became_active":"2023-10-06T04:32:23.452236+0000","last_became_peered":"2023-10-06T04:32:23.452236+0000","last_unstale":"2023-10-06T04:32:34.268764+0000","last_undegraded":"2023-10-06T04:32:34.268764+0000","last_fullsized":"2023-10-06T04:32:34.268764+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.c","version":"34'6","reported_seq":18,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269934+0000","last_change":"2023-10-06T04:32:24.265984+0000","last_active":"2023-10-06T04:32:34.269934+0000","last_peered":"2023-10-06T04:32:34.269934+0000","last_clean":"2023-10-06T04:32:34.269934+0000","last_became_active":"2023-10-06T04:32:24.265707+0000","last_became_peered":"2023-10-06T04:32:24.265707+0000","last_unstale":"2023-10-06T04:32:34.269934+0000","last_undegraded":"2023-10-06T04:32:34.269934+0000","last_fullsized":"2023-10-06T04:32:34.269934+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'2","reported_seq":16,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315804+0000","last_change":"2023-10-06T04:32:23.447020+0000","last_active":"2023-10-06T04:32:35.315804+0000","last_peered":"2023-10-06T04:32:35.315804+0000","last_clean":"2023-10-06T04:32:35.315804+0000","last_became_active":"2023-10-06T04:32:23.446786+0000","last_became_peered":"2023-10-06T04:32:23.446786+0000","last_unstale":"2023-10-06T04:32:35.315804+0000","last_undegraded":"2023-10-06T04:32:35.315804+0000","last_fullsized":"2023-10-06T04:32:35.315804+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.805674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.d","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315600+0000","last_change":"2023-10-06T04:32:24.265722+0000","last_active":"2023-10-06T04:32:35.315600+0000","last_peered":"2023-10-06T04:32:35.315600+0000","last_clean":"2023-10-06T04:32:35.315600+0000","last_became_active":"2023-10-06T04:32:24.265435+0000","last_became_peered":"2023-10-06T04:32:24.265435+0000","last_unstale":"2023-10-06T04:32:35.315600+0000","last_undegraded":"2023-10-06T04:32:35.315600+0000","last_fullsized":"2023-10-06T04:32:35.315600+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.261738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'3","reported_seq":16,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260867+0000","last_change":"2023-10-06T04:32:23.442348+0000","last_active":"2023-10-06T04:32:34.260867+0000","last_peered":"2023-10-06T04:32:34.260867+0000","last_clean":"2023-10-06T04:32:34.260867+0000","last_became_active":"2023-10-06T04:32:23.441833+0000","last_became_peered":"2023-10-06T04:32:23.441833+0000","last_unstale":"2023-10-06T04:32:34.260867+0000","last_undegraded":"2023-10-06T04:32:34.260867+0000","last_fullsized":"2023-10-06T04:32:34.260867+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.916159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.e","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260744+0000","last_change":"2023-10-06T04:32:24.254622+0000","last_active":"2023-10-06T04:32:34.260744+0000","last_peered":"2023-10-06T04:32:34.260744+0000","last_clean":"2023-10-06T04:32:34.260744+0000","last_became_active":"2023-10-06T04:32:24.254294+0000","last_became_peered":"2023-10-06T04:32:24.254294+0000","last_unstale":"2023-10-06T04:32:34.260744+0000","last_undegraded":"2023-10-06T04:32:34.260744+0000","last_fullsized":"2023-10-06T04:32:34.260744+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'2","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263152+0000","last_change":"2023-10-06T04:32:23.447777+0000","last_active":"2023-10-06T04:32:34.263152+0000","last_peered":"2023-10-06T04:32:34.263152+0000","last_clean":"2023-10-06T04:32:34.263152+0000","last_became_active":"2023-10-06T04:32:23.447526+0000","last_became_peered":"2023-10-06T04:32:23.447526+0000","last_unstale":"2023-10-06T04:32:34.263152+0000","last_undegraded":"2023-10-06T04:32:34.263152+0000","last_fullsized":"2023-10-06T04:32:34.263152+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:59.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.f","version":"34'9","reported_seq":23,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263255+0000","last_change":"2023-10-06T04:32:24.270632+0000","last_active":"2023-10-06T04:32:34.263255+0000","last_peered":"2023-10-06T04:32:34.263255+0000","last_clean":"2023-10-06T04:32:34.263255+0000","last_became_active":"2023-10-06T04:32:24.270263+0000","last_became_peered":"2023-10-06T04:32:24.270263+0000","last_unstale":"2023-10-06T04:32:34.263255+0000","last_undegraded":"2023-10-06T04:32:34.263255+0000","last_fullsized":"2023-10-06T04:32:34.263255+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:19.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268936+0000","last_change":"2023-10-06T04:32:23.446504+0000","last_active":"2023-10-06T04:32:34.268936+0000","last_peered":"2023-10-06T04:32:34.268936+0000","last_clean":"2023-10-06T04:32:34.268936+0000","last_became_active":"2023-10-06T04:32:23.446221+0000","last_became_peered":"2023-10-06T04:32:23.446221+0000","last_unstale":"2023-10-06T04:32:34.268936+0000","last_undegraded":"2023-10-06T04:32:34.268936+0000","last_fullsized":"2023-10-06T04:32:34.268936+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.8","version":"34'9","reported_seq":22,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268647+0000","last_change":"2023-10-06T04:32:24.271430+0000","last_active":"2023-10-06T04:32:34.268647+0000","last_peered":"2023-10-06T04:32:34.268647+0000","last_clean":"2023-10-06T04:32:34.268647+0000","last_became_active":"2023-10-06T04:32:24.271180+0000","last_became_peered":"2023-10-06T04:32:24.271180+0000","last_unstale":"2023-10-06T04:32:34.268647+0000","last_undegraded":"2023-10-06T04:32:34.268647+0000","last_fullsized":"2023-10-06T04:32:34.268647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'4","reported_seq":19,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263236+0000","last_change":"2023-10-06T04:32:23.445911+0000","last_active":"2023-10-06T04:32:34.263236+0000","last_peered":"2023-10-06T04:32:34.263236+0000","last_clean":"2023-10-06T04:32:34.263236+0000","last_became_active":"2023-10-06T04:32:23.445582+0000","last_became_peered":"2023-10-06T04:32:23.445582+0000","last_unstale":"2023-10-06T04:32:34.263236+0000","last_undegraded":"2023-10-06T04:32:34.263236+0000","last_fullsized":"2023-10-06T04:32:34.263236+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:35.404671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.9","version":"34'2","reported_seq":14,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269245+0000","last_change":"2023-10-06T04:32:24.259250+0000","last_active":"2023-10-06T04:32:34.269245+0000","last_peered":"2023-10-06T04:32:34.269245+0000","last_clean":"2023-10-06T04:32:34.269245+0000","last_became_active":"2023-10-06T04:32:24.258962+0000","last_became_peered":"2023-10-06T04:32:24.258962+0000","last_unstale":"2023-10-06T04:32:34.269245+0000","last_undegraded":"2023-10-06T04:32:34.269245+0000","last_fullsized":"2023-10-06T04:32:34.269245+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262972+0000","last_change":"2023-10-06T04:32:23.445529+0000","last_active":"2023-10-06T04:32:34.262972+0000","last_peered":"2023-10-06T04:32:34.262972+0000","last_clean":"2023-10-06T04:32:34.262972+0000","last_became_active":"2023-10-06T04:32:23.445245+0000","last_became_peered":"2023-10-06T04:32:23.445245+0000","last_unstale":"2023-10-06T04:32:34.262972+0000","last_undegraded":"2023-10-06T04:32:34.262972+0000","last_fullsized":"2023-10-06T04:32:34.262972+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.a","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315277+0000","last_change":"2023-10-06T04:32:24.270658+0000","last_active":"2023-10-06T04:32:35.315277+0000","last_peered":"2023-10-06T04:32:35.315277+0000","last_clean":"2023-10-06T04:32:35.315277+0000","last_became_active":"2023-10-06T04:32:24.270394+0000","last_became_peered":"2023-10-06T04:32:24.270394+0000","last_unstale":"2023-10-06T04:32:35.315277+0000","last_undegraded":"2023-10-06T04:32:35.315277+0000","last_fullsized":"2023-10-06T04:32:35.315277+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269283+0000","last_change":"2023-10-06T04:32:23.452178+0000","last_active":"2023-10-06T04:32:34.269283+0000","last_peered":"2023-10-06T04:32:34.269283+0000","last_clean":"2023-10-06T04:32:34.269283+0000","last_became_active":"2023-10-06T04:32:23.451849+0000","last_became_peered":"2023-10-06T04:32:23.451849+0000","last_unstale":"2023-10-06T04:32:34.269283+0000","last_undegraded":"2023-10-06T04:32:34.269283+0000","last_fullsized":"2023-10-06T04:32:34.269283+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.026875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.b","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269679+0000","last_change":"2023-10-06T04:32:24.265320+0000","last_active":"2023-10-06T04:32:34.269679+0000","last_peered":"2023-10-06T04:32:34.269679+0000","last_clean":"2023-10-06T04:32:34.269679+0000","last_became_active":"2023-10-06T04:32:24.264894+0000","last_became_peered":"2023-10-06T04:32:24.264894+0000","last_unstale":"2023-10-06T04:32:34.269679+0000","last_undegraded":"2023-10-06T04:32:34.269679+0000","last_fullsized":"2023-10-06T04:32:34.269679+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262902+0000","last_change":"2023-10-06T04:32:23.444136+0000","last_active":"2023-10-06T04:32:34.262902+0000","last_peered":"2023-10-06T04:32:34.262902+0000","last_clean":"2023-10-06T04:32:34.262902+0000","last_became_active":"2023-10-06T04:32:23.443839+0000","last_became_peered":"2023-10-06T04:32:23.443839+0000","last_unstale":"2023-10-06T04:32:34.262902+0000","last_undegraded":"2023-10-06T04:32:34.262902+0000","last_fullsized":"2023-10-06T04:32:34.262902+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:10.814479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.0","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268848+0000","last_change":"2023-10-06T04:32:24.269672+0000","last_active":"2023-10-06T04:32:34.268848+0000","last_peered":"2023-10-06T04:32:34.268848+0000","last_clean":"2023-10-06T04:32:34.268848+0000","last_became_active":"2023-10-06T04:32:24.269407+0000","last_became_peered":"2023-10-06T04:32:24.269407+0000","last_unstale":"2023-10-06T04:32:34.268848+0000","last_undegraded":"2023-10-06T04:32:34.268848+0000","last_fullsized":"2023-10-06T04:32:34.268848+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269617+0000","last_change":"2023-10-06T04:32:23.446685+0000","last_active":"2023-10-06T04:32:34.269617+0000","last_peered":"2023-10-06T04:32:34.269617+0000","last_clean":"2023-10-06T04:32:34.269617+0000","last_became_active":"2023-10-06T04:32:23.446432+0000","last_became_peered":"2023-10-06T04:32:23.446432+0000","last_unstale":"2023-10-06T04:32:34.269617+0000","last_undegraded":"2023-10-06T04:32:34.269617+0000","last_fullsized":"2023-10-06T04:32:34.269617+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268778+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:32:34.268778+0000","last_peered":"2023-10-06T04:32:34.268778+0000","last_clean":"2023-10-06T04:32:34.268778+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:32:34.268778+0000","last_undegraded":"2023-10-06T04:32:34.268778+0000","last_fullsized":"2023-10-06T04:32:34.268778+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.6","version":"34'7","reported_seq":19,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270090+0000","last_change":"2023-10-06T04:32:24.266084+0000","last_active":"2023-10-06T04:32:34.270090+0000","last_peered":"2023-10-06T04:32:34.270090+0000","last_clean":"2023-10-06T04:32:34.270090+0000","last_became_active":"2023-10-06T04:32:24.265847+0000","last_became_peered":"2023-10-06T04:32:24.265847+0000","last_unstale":"2023-10-06T04:32:34.270090+0000","last_undegraded":"2023-10-06T04:32:34.270090+0000","last_fullsized":"2023-10-06T04:32:34.270090+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.270247+0000","last_change":"2023-10-06T04:32:23.451106+0000","last_active":"2023-10-06T04:32:34.270247+0000","last_peered":"2023-10-06T04:32:34.270247+0000","last_clean":"2023-10-06T04:32:34.270247+0000","last_became_active":"2023-10-06T04:32:23.450838+0000","last_became_peered":"2023-10-06T04:32:23.450838+0000","last_unstale":"2023-10-06T04:32:34.270247+0000","last_undegraded":"2023-10-06T04:32:34.270247+0000","last_fullsized":"2023-10-06T04:32:34.270247+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315391+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:32:35.315391+0000","last_peered":"2023-10-06T04:32:35.315391+0000","last_clean":"2023-10-06T04:32:35.315391+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:32:35.315391+0000","last_undegraded":"2023-10-06T04:32:35.315391+0000","last_fullsized":"2023-10-06T04:32:35.315391+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.5","version":"34'5","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269578+0000","last_change":"2023-10-06T04:32:24.272289+0000","last_active":"2023-10-06T04:32:34.269578+0000","last_peered":"2023-10-06T04:32:34.269578+0000","last_clean":"2023-10-06T04:32:34.269578+0000","last_became_active":"2023-10-06T04:32:24.271996+0000","last_became_peered":"2023-10-06T04:32:24.271996+0000","last_unstale":"2023-10-06T04:32:34.269578+0000","last_undegraded":"2023-10-06T04:32:34.269578+0000","last_fullsized":"2023-10-06T04:32:34.269578+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'3","reported_seq":18,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.263158+0000","last_change":"2023-10-06T04:32:23.444286+0000","last_active":"2023-10-06T04:32:34.263158+0000","last_peered":"2023-10-06T04:32:34.263158+0000","last_clean":"2023-10-06T04:32:34.263158+0000","last_became_active":"2023-10-06T04:32:23.443982+0000","last_became_peered":"2023-10-06T04:32:23.443982+0000","last_unstale":"2023-10-06T04:32:34.263158+0000","last_undegraded":"2023-10-06T04:32:34.263158+0000","last_fullsized":"2023-10-06T04:32:34.263158+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:20.394340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":28,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.262988+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:32:34.262988+0000","last_peered":"2023-10-06T04:32:34.262988+0000","last_clean":"2023-10-06T04:32:34.262988+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:32:34.262988+0000","last_undegraded":"2023-10-06T04:32:34.262988+0000","last_fullsized":"2023-10-06T04:32:34.262988+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.7","version":"34'8","reported_seq":21,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315767+0000","last_change":"2023-10-06T04:32:24.265180+0000","last_active":"2023-10-06T04:32:35.315767+0000","last_peered":"2023-10-06T04:32:35.315767+0000","last_clean":"2023-10-06T04:32:35.315767+0000","last_became_active":"2023-10-06T04:32:24.264769+0000","last_became_peered":"2023-10-06T04:32:24.264769+0000","last_unstale":"2023-10-06T04:32:35.315767+0000","last_undegraded":"2023-10-06T04:32:35.315767+0000","last_fullsized":"2023-10-06T04:32:35.315767+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":14,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268997+0000","last_change":"2023-10-06T04:32:23.452281+0000","last_active":"2023-10-06T04:32:34.268997+0000","last_peered":"2023-10-06T04:32:34.268997+0000","last_clean":"2023-10-06T04:32:34.268997+0000","last_became_active":"2023-10-06T04:32:23.452023+0000","last_became_peered":"2023-10-06T04:32:23.452023+0000","last_unstale":"2023-10-06T04:32:34.268997+0000","last_undegraded":"2023-10-06T04:32:34.268997+0000","last_fullsized":"2023-10-06T04:32:34.268997+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":25,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315453+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:32:35.315453+0000","last_peered":"2023-10-06T04:32:35.315453+0000","last_clean":"2023-10-06T04:32:35.315453+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:32:35.315453+0000","last_undegraded":"2023-10-06T04:32:35.315453+0000","last_fullsized":"2023-10-06T04:32:35.315453+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":179,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269424+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:32:34.269424+0000","last_peered":"2023-10-06T04:32:34.269424+0000","last_clean":"2023-10-06T04:32:34.269424+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:32:34.269424+0000","last_undegraded":"2023-10-06T04:32:34.269424+0000","last_fullsized":"2023-10-06T04:32:34.269424+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.4","version":"34'9","reported_seq":21,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268400+0000","last_change":"2023-10-06T04:32:24.267047+0000","last_active":"2023-10-06T04:32:34.268400+0000","last_peered":"2023-10-06T04:32:34.268400+0000","last_clean":"2023-10-06T04:32:34.268400+0000","last_became_active":"2023-10-06T04:32:24.266777+0000","last_became_peered":"2023-10-06T04:32:24.266777+0000","last_unstale":"2023-10-06T04:32:34.268400+0000","last_undegraded":"2023-10-06T04:32:34.268400+0000","last_fullsized":"2023-10-06T04:32:34.268400+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'3","reported_seq":17,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:35.315859+0000","last_change":"2023-10-06T04:32:23.448480+0000","last_active":"2023-10-06T04:32:35.315859+0000","last_peered":"2023-10-06T04:32:35.315859+0000","last_clean":"2023-10-06T04:32:35.315859+0000","last_became_active":"2023-10-06T04:32:23.448242+0000","last_became_peered":"2023-10-06T04:32:23.448242+0000","last_unstale":"2023-10-06T04:32:35.315859+0000","last_undegraded":"2023-10-06T04:32:35.315859+0000","last_fullsized":"2023-10-06T04:32:35.315859+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269418+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:32:34.269418+0000","last_peered":"2023-10-06T04:32:34.269418+0000","last_clean":"2023-10-06T04:32:34.269418+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:32:34.269418+0000","last_undegraded":"2023-10-06T04:32:34.269418+0000","last_fullsized":"2023-10-06T04:32:34.269418+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.1","version":"34'5","reported_seq":17,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268375+0000","last_change":"2023-10-06T04:32:24.266050+0000","last_active":"2023-10-06T04:32:34.268375+0000","last_peered":"2023-10-06T04:32:34.268375+0000","last_clean":"2023-10-06T04:32:34.268375+0000","last_became_active":"2023-10-06T04:32:24.265769+0000","last_became_peered":"2023-10-06T04:32:24.265769+0000","last_unstale":"2023-10-06T04:32:34.268375+0000","last_undegraded":"2023-10-06T04:32:34.268375+0000","last_fullsized":"2023-10-06T04:32:34.268375+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:01.427293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":15,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268726+0000","last_change":"2023-10-06T04:32:23.443114+0000","last_active":"2023-10-06T04:32:34.268726+0000","last_peered":"2023-10-06T04:32:34.268726+0000","last_clean":"2023-10-06T04:32:34.268726+0000","last_became_active":"2023-10-06T04:32:23.442844+0000","last_became_peered":"2023-10-06T04:32:23.442844+0000","last_unstale":"2023-10-06T04:32:34.268726+0000","last_undegraded":"2023-10-06T04:32:34.268726+0000","last_fullsized":"2023-10-06T04:32:34.268726+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":26,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269667+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:32:34.269667+0000","last_peered":"2023-10-06T04:32:34.269667+0000","last_clean":"2023-10-06T04:32:34.269667+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:32:34.269667+0000","last_undegraded":"2023-10-06T04:32:34.269667+0000","last_fullsized":"2023-10-06T04:32:34.269667+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.2","version":"34'8","reported_seq":20,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.261395+0000","last_change":"2023-10-06T04:32:24.265505+0000","last_active":"2023-10-06T04:32:34.261395+0000","last_peered":"2023-10-06T04:32:34.261395+0000","last_clean":"2023-10-06T04:32:34.261395+0000","last_became_active":"2023-10-06T04:32:24.265228+0000","last_became_peered":"2023-10-06T04:32:24.265228+0000","last_unstale":"2023-10-06T04:32:34.261395+0000","last_undegraded":"2023-10-06T04:32:34.261395+0000","last_fullsized":"2023-10-06T04:32:34.261395+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.260966+0000","last_change":"2023-10-06T04:32:23.438229+0000","last_active":"2023-10-06T04:32:34.260966+0000","last_peered":"2023-10-06T04:32:34.260966+0000","last_clean":"2023-10-06T04:32:34.260966+0000","last_became_active":"2023-10-06T04:32:23.437954+0000","last_became_peered":"2023-10-06T04:32:23.437954+0000","last_unstale":"2023-10-06T04:32:34.260966+0000","last_undegraded":"2023-10-06T04:32:34.260966+0000","last_fullsized":"2023-10-06T04:32:34.260966+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:22.972008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":26,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.268724+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:32:34.268724+0000","last_peered":"2023-10-06T04:32:34.268724+0000","last_clean":"2023-10-06T04:32:34.268724+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:32:34.268724+0000","last_undegraded":"2023-10-06T04:32:34.268724+0000","last_fullsized":"2023-10-06T04:32:34.268724+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-10-06T04:32:36.583838+0000","last_change":"2023-10-06T04:32:36.583838+0000","last_active":"2023-10-06T04:32:36.583838+0000","last_peered":"2023-10-06T04:32:36.583838+0000","last_clean":"2023-10-06T04:32:36.583838+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-10-06T04:32:36.583838+0000","last_undegraded":"2023-10-06T04:32:36.583838+0000","last_fullsized":"2023-10-06T04:32:36.583838+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"6.3","version":"34'7","reported_seq":20,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269083+0000","last_change":"2023-10-06T04:32:24.272223+0000","last_active":"2023-10-06T04:32:34.269083+0000","last_peered":"2023-10-06T04:32:34.269083+0000","last_clean":"2023-10-06T04:32:34.269083+0000","last_became_active":"2023-10-06T04:32:24.271905+0000","last_became_peered":"2023-10-06T04:32:24.271905+0000","last_unstale":"2023-10-06T04:32:34.269083+0000","last_undegraded":"2023-10-06T04:32:34.269083+0000","last_fullsized":"2023-10-06T04:32:34.269083+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:17.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":13,"reported_epoch":37,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269275+0000","last_change":"2023-10-06T04:32:23.444321+0000","last_active":"2023-10-06T04:32:34.269275+0000","last_peered":"2023-10-06T04:32:34.269275+0000","last_clean":"2023-10-06T04:32:34.269275+0000","last_became_active":"2023-10-06T04:32:23.444073+0000","last_became_peered":"2023-10-06T04:32:23.444073+0000","last_unstale":"2023-10-06T04:32:34.269275+0000","last_undegraded":"2023-10-06T04:32:34.269275+0000","last_fullsized":"2023-10-06T04:32:34.269275+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:37.675115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":27,"reported_epoch":38,"state":"active+clean","last_fresh":"2023-10-06T04:32:34.269325+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:32:34.269325+0000","last_peered":"2023-10-06T04:32:34.269325+0000","last_clean":"2023-10-06T04:32:34.269325+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:32:34.269325+0000","last_undegraded":"2023-10-06T04:32:34.269325+0000","last_fullsized":"2023-10-06T04:32:34.269325+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574867,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300340,"kb_used_data":3052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071500,"statfs":{"total":96636764160,"available":96329216000,"internally_reserved":0,"allocated":3125248,"data_stored":2869386,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.33700000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.90500000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.61199999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67000000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.85599999999999998},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.54000000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.95899999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.59199999999999997}]}]},{"osd":6,"up_from":13,"seq":55834574866,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":188202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.41199999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54000000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.63200000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.496},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.57599999999999996}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.81100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297700,"kb_used_data":412,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074140,"statfs":{"total":96636764160,"available":96331919360,"internally_reserved":0,"allocated":421888,"data_stored":175530,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.33000000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.40100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.36599999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.86199999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.93600000000000005},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.64900000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.67200000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.70199999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.66300000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.88600000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574867,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298304,"kb_used_data":1016,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073536,"statfs":{"total":96636764160,"available":96331300864,"internally_reserved":0,"allocated":1040384,"data_stored":778570,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.26100000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.34100000000000003},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.29299999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.27200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.53700000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.48099999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.52300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.56499999999999995}]}]},{"osd":2,"up_from":13,"seq":55834574867,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297712,"kb_used_data":424,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074128,"statfs":{"total":96636764160,"available":96331907072,"internally_reserved":0,"allocated":434176,"data_stored":181866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.40600000000000003},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.23100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.26200000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.46000000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.371}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47299999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.48199999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.499}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55900000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.57099999999999995}]}]},{"osd":3,"up_from":13,"seq":55834574867,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301848,"kb_used_data":4532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069992,"statfs":{"total":96636764160,"available":96327671808,"internally_reserved":0,"allocated":4640768,"data_stored":4382506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.248},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.29999999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.32600000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41099999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.61699999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.52900000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.51900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.54400000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574868,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297724,"kb_used_data":436,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074116,"statfs":{"total":96636764160,"available":96331894784,"internally_reserved":0,"allocated":446464,"data_stored":188202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.52300000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45300000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.50800000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.57099999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.47899999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.495}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.36299999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.54900000000000004}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54000000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303844,"kb_used_data":6556,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067996,"statfs":{"total":96636764160,"available":96325627904,"internally_reserved":0,"allocated":6713344,"data_stored":6466986,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.45800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.35499999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.56899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.52200000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.58399999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.66700000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.64300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.626}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:32:42.415 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1d is not active+clean 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1c is not active+clean 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1f is not active+clean 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1e is not active+clean 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.19 is not active+clean 2023-10-06T04:32:42.416 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.18 is not active+clean 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1b is not active+clean 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1a is not active+clean 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.15 is not active+clean 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.417 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.14 is not active+clean 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.17 is not active+clean 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.16 is not active+clean 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.11 is not active+clean 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.418 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.10 is not active+clean 2023-10-06T04:32:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.13 is not active+clean 2023-10-06T04:32:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.12 is not active+clean 2023-10-06T04:32:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.d is not active+clean 2023-10-06T04:32:42.419 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.c is not active+clean 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.f is not active+clean 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.e is not active+clean 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.9 is not active+clean 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.8 is not active+clean 2023-10-06T04:32:42.420 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.b is not active+clean 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.a is not active+clean 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.1 is not active+clean 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.7 is not active+clean 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.421 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.4 is not active+clean 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.6 is not active+clean 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.5 is not active+clean 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.0 is not active+clean 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.3 is not active+clean 2023-10-06T04:32:42.422 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.423 INFO:tasks.cephfs.filesystem.ceph_manager:PG 7.2 is not active+clean 2023-10-06T04:32:42.423 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '7.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-10-06T04:32:36.583838+0000', 'last_change': '2023-10-06T04:32:36.583838+0000', 'last_active': '2023-10-06T04:32:36.583838+0000', 'last_peered': '2023-10-06T04:32:36.583838+0000', 'last_clean': '2023-10-06T04:32:36.583838+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-10-06T04:32:36.583838+0000', 'last_undegraded': '2023-10-06T04:32:36.583838+0000', 'last_fullsized': '2023-10-06T04:32:36.583838+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'last_clean_scrub_stamp': '2023-10-06T04:32:36.583838+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-10-06T04:32:42.423 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-10-06T04:32:42.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.561+0000 7fbb185cb700 1 Processor -- start 2023-10-06T04:32:42.563 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.561+0000 7fbb185cb700 1 -- start start 2023-10-06T04:32:42.565 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb185cb700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbb101560c0 0x7fbb100c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.565 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb185cb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006e960 0x7fbb100ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb185cb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb100c8580 con 0x7fbb1006f080 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb185cb700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb100c8860 con 0x7fbb101560c0 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb185cb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb100c8b40 con 0x7fbb1006e960 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbb101560c0 0x7fbb100c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbb101560c0 0x7fbb100c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33626/0 (socket says 172.21.15.123:33626) 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb15b66700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fbb1006f080 0x7fbb1006e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45056/0 (socket says 172.21.15.123:45056) 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 -- 172.21.15.123:0/2978388408 learned_addr learned my addr 172.21.15.123:0/2978388408 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16b68700 1 --2- 172.21.15.123:0/2978388408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006e960 0x7fbb100ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 -- 172.21.15.123:0/2978388408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006e960 msgr2=0x7fbb100ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 --2- 172.21.15.123:0/2978388408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006e960 0x7fbb100ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 -- 172.21.15.123:0/2978388408 >> v1:172.21.15.123:6789/0 conn(0x7fbb1006f080 legacy=0x7fbb1006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 -- 172.21.15.123:0/2978388408 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb100c8e20 con 0x7fbb101560c0 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb15365700 1 -- 172.21.15.123:0/2978388408 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1707637909 0 0) 0x7fbb100c8580 con 0x7fbb1006f080 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb16367700 1 --2- 172.21.15.123:0/2978388408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb101560c0 0x7fbb100c7fa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0099b0 tx=0x7fbb0c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e90192bb2893c22 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb15365700 1 -- 172.21.15.123:0/2978388408 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00b860 con 0x7fbb101560c0 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.562+0000 7fbb15365700 1 -- 172.21.15.123:0/2978388408 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c00b9c0 con 0x7fbb101560c0 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb15365700 1 -- 172.21.15.123:0/2978388408 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c00bd60 con 0x7fbb101560c0 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 -- 172.21.15.123:0/2978388408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb101560c0 msgr2=0x7fbb100c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 --2- 172.21.15.123:0/2978388408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb101560c0 0x7fbb100c7fa0 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c0099b0 tx=0x7fbb0c009250 comp rx=0 tx=0).stop 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 -- 172.21.15.123:0/2978388408 shutdown_connections 2023-10-06T04:32:42.567 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 --2- 172.21.15.123:0/2978388408 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006e960 0x7fbb100ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 --2- 172.21.15.123:0/2978388408 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb101560c0 0x7fbb100c7fa0 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 -- 172.21.15.123:0/2978388408 >> 172.21.15.123:0/2978388408 conn(0x7fbb100cda80 msgr2=0x7fbb100c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:42.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 -- 172.21.15.123:0/2978388408 shutdown_connections 2023-10-06T04:32:42.568 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.563+0000 7fbb185cb700 1 -- 172.21.15.123:0/2978388408 wait complete. 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.564+0000 7fbb185cb700 1 Processor -- start 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.564+0000 7fbb185cb700 1 -- start start 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb185cb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1006e960 0x7fbb100abc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb185cb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006f080 0x7fbb10067e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb185cb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb101560c0 0x7fbb1014f1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb185cb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1016dee0 con 0x7fbb1006e960 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb185cb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1016d310 con 0x7fbb101560c0 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb185cb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb1016d5b0 con 0x7fbb1006f080 2023-10-06T04:32:42.571 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1006e960 0x7fbb100abc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1006e960 0x7fbb100abc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33636/0 (socket says 172.21.15.123:33636) 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 -- 172.21.15.123:0/295453895 learned_addr learned my addr 172.21.15.123:0/295453895 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 -- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006f080 msgr2=0x7fbb10067e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb15b66700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006f080 0x7fbb10067e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16b68700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb101560c0 0x7fbb1014f1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006f080 0x7fbb10067e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 -- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb101560c0 msgr2=0x7fbb1014f1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb101560c0 0x7fbb1014f1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb16367700 1 -- 172.21.15.123:0/295453895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb0c0094f0 con 0x7fbb1006e960 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.565+0000 7fbb15b66700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006f080 0x7fbb10067e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.566+0000 7fbb16367700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1006e960 0x7fbb100abc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.572 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.566+0000 7fbb067fc700 1 -- 172.21.15.123:0/295453895 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c01a420 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.566+0000 7fbb067fc700 1 -- 172.21.15.123:0/295453895 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb0c01a580 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.566+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb1016d890 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.566+0000 7fbb067fc700 1 -- 172.21.15.123:0/295453895 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb0c01a940 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.566+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb100ac1b0 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.567+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb1016e1c0 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.567+0000 7fbb067fc700 1 -- 172.21.15.123:0/295453895 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbb0c01acf0 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.567+0000 7fbb067fc700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbaf4066f30 0x7fbaf40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.567+0000 7fbb067fc700 1 -- 172.21.15.123:0/295453895 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbb0c0862f0 con 0x7fbb1006e960 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.567+0000 7fbb15b66700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbaf4066f30 0x7fbaf40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.573 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.568+0000 7fbb15b66700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbaf4066f30 0x7fbaf40693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb10159b90 tx=0x7fbb00009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.574 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.572+0000 7fbb067fc700 1 -- 172.21.15.123:0/295453895 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbb0c0531e0 con 0x7fbb1006e960 2023-10-06T04:32:42.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.826+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7fbb100cde80 con 0x7fbb1006e960 2023-10-06T04:32:42.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.827+0000 7fbb067fc700 1 -- 172.21.15.123:0/295453895 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1862 (secure 0 0 0) 0x7fbb0c056800 con 0x7fbb1006e960 2023-10-06T04:32:42.830 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:42.830 INFO:teuthology.orchestra.run.smithi123.stdout:{"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","health":{"status":"HEALTH_ERR","checks":{"FS_DEGRADED":{"severity":"HEALTH_WARN","summary":{"message":"2 filesystems are degraded","count":2},"muted":false},"MDS_ALL_DOWN":{"severity":"HEALTH_ERR","summary":{"message":"2 filesystems are offline","count":2},"muted":false}},"mutes":[]},"election_epoch":4,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":93,"monmap":{"epoch":1,"min_mon_release_name":"quincy","num_mons":3},"osdmap":{"epoch":42,"num_osds":8,"num_up_osds":8,"osd_up_since":1696566680,"num_in_osds":8,"osd_in_since":1696566672,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"active+clean","count":105},{"state_name":"unknown","count":32}],"num_pgs":137,"num_pools":5,"num_objects":4,"data_bytes":6881824,"bytes_used":2452680704,"bytes_avail":770641432576,"bytes_total":773094113280,"unknown_pgs_ratio":0.23357664048671722,"read_bytes_sec":0,"write_bytes_sec":1052319,"read_op_per_sec":0,"write_op_per_sec":8},"fsmap":{"epoch":22,"id":2,"up":0,"in":1,"max":1,"id":3,"up":0,"in":1,"max":1,"by_rank":[],"up:standby":4,"failed":2},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-10-06T04:32:22.899038+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbaf4066f30 msgr2=0x7fbaf40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbaf4066f30 0x7fbaf40693e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7fbb10159b90 tx=0x7fbb00009040 comp rx=0 tx=0).stop 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1006e960 msgr2=0x7fbb100abc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1006e960 0x7fbb100abc70 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fbb0c006fd0 tx=0x7fbb0c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 shutdown_connections 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbaf4066f30 0x7fbaf40693e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbb1006e960 0x7fbb100abc70 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.832 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbb101560c0 0x7fbb1014f1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 --2- 172.21.15.123:0/295453895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbb1006f080 0x7fbb10067e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 >> 172.21.15.123:0/295453895 conn(0x7fbb100cda80 msgr2=0x7fbb100ae900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:42.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 shutdown_connections 2023-10-06T04:32:42.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.829+0000 7fbb185cb700 1 -- 172.21.15.123:0/295453895 wait complete. 2023-10-06T04:32:42.843 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-10-06T04:32:42.844 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:32:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.959+0000 7fbfbd925700 1 Processor -- start 2023-10-06T04:32:42.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.959+0000 7fbfbd925700 1 -- start start 2023-10-06T04:32:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfbd925700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbfb80ac720 0x7fbfb80ae780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfbd925700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fbfb80680b0 0x7fbfb80acaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfbd925700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbfb806b760 con 0x7fbfb806e250 2023-10-06T04:32:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfbd925700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb806deb0 con 0x7fbfb80ac720 2023-10-06T04:32:42.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfbd925700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbfb80aecc0 con 0x7fbfb80680b0 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfb6ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbfb80ac720 0x7fbfb80ae780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfb6ffd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fbfb80ac720 0x7fbfb80ae780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33642/0 (socket says 172.21.15.123:33642) 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/2028895163 learned_addr learned my addr 172.21.15.123:0/2028895163 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.960+0000 7fbfb67fc700 1 --2- 172.21.15.123:0/2028895163 >> v2:172.21.15.177:3300/0 conn(0x7fbfb80680b0 0x7fbfb80acaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/2028895163 >> v2:172.21.15.177:3300/0 conn(0x7fbfb80680b0 msgr2=0x7fbfb80acaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb6ffd700 1 --2- 172.21.15.123:0/2028895163 >> v2:172.21.15.177:3300/0 conn(0x7fbfb80680b0 0x7fbfb80acaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/2028895163 >> v1:172.21.15.177:6789/0 conn(0x7fbfb806e250 legacy=0x7fbfb806e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/2028895163 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfb80ad030 con 0x7fbfb80ac720 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb67fc700 1 --2- 172.21.15.123:0/2028895163 >> v2:172.21.15.177:3300/0 conn(0x7fbfb80680b0 0x7fbfb80acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:42.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb6ffd700 1 --2- 172.21.15.123:0/2028895163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80ac720 0x7fbfb80ae780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbfb806cf00 tx=0x7fbfac009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=901f56acb26bb2bd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb5ffb700 1 -- 172.21.15.123:0/2028895163 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00b870 con 0x7fbfb80ac720 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb5ffb700 1 -- 172.21.15.123:0/2028895163 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac00b9d0 con 0x7fbfb80ac720 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfb5ffb700 1 -- 172.21.15.123:0/2028895163 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac00bd70 con 0x7fbfb80ac720 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfbd925700 1 -- 172.21.15.123:0/2028895163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80ac720 msgr2=0x7fbfb80ae780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfbd925700 1 --2- 172.21.15.123:0/2028895163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80ac720 0x7fbfb80ae780 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fbfb806cf00 tx=0x7fbfac009250 comp rx=0 tx=0).stop 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfbd925700 1 -- 172.21.15.123:0/2028895163 shutdown_connections 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfbd925700 1 --2- 172.21.15.123:0/2028895163 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80ac720 0x7fbfb80ae780 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfbd925700 1 --2- 172.21.15.123:0/2028895163 >> v2:172.21.15.177:3300/0 conn(0x7fbfb80680b0 0x7fbfb80acaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfbd925700 1 -- 172.21.15.123:0/2028895163 >> 172.21.15.123:0/2028895163 conn(0x7fbfb80cda80 msgr2=0x7fbfb80c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:42.966 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.961+0000 7fbfbd925700 1 -- 172.21.15.123:0/2028895163 shutdown_connections 2023-10-06T04:32:42.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.962+0000 7fbfbd925700 1 -- 172.21.15.123:0/2028895163 wait complete. 2023-10-06T04:32:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 Processor -- start 2023-10-06T04:32:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 -- start start 2023-10-06T04:32:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80680b0 0x7fbfb80c0a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfb806e250 0x7fbfb81591f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfb80ac720 0x7fbfb8156570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb8159730 con 0x7fbfb80680b0 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80c10a0 con 0x7fbfb80ac720 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.963+0000 7fbfbd925700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbfb80c1380 con 0x7fbfb806e250 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80680b0 0x7fbfb80c0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80680b0 0x7fbfb80c0a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33644/0 (socket says 172.21.15.123:33644) 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/3827903003 learned_addr learned my addr 172.21.15.123:0/3827903003 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/3827903003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfb806e250 msgr2=0x7fbfb81591f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfb806e250 0x7fbfb81591f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/3827903003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfb80ac720 msgr2=0x7fbfb8156570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfb80ac720 0x7fbfb8156570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 -- 172.21.15.123:0/3827903003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbfac0094f0 con 0x7fbfb80680b0 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfb6ffd700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80680b0 0x7fbfb80c0a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0030c0 tx=0x7fbfac016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbf9effd700 1 -- 172.21.15.123:0/3827903003 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac01b680 con 0x7fbfb80680b0 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbf9effd700 1 -- 172.21.15.123:0/3827903003 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbfac01b7e0 con 0x7fbfb80680b0 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbfb80c1660 con 0x7fbfb80680b0 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbf9effd700 1 -- 172.21.15.123:0/3827903003 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbfac01bb80 con 0x7fbfb80680b0 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.964+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbfb8156ab0 con 0x7fbfb80680b0 2023-10-06T04:32:42.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.965+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbfb80cde80 con 0x7fbfb80680b0 2023-10-06T04:32:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.965+0000 7fbf9effd700 1 -- 172.21.15.123:0/3827903003 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fbfac027460 con 0x7fbfb80680b0 2023-10-06T04:32:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.966+0000 7fbf9effd700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbfa4066f30 0x7fbfa40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.966+0000 7fbf9effd700 1 -- 172.21.15.123:0/3827903003 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fbfac0867e0 con 0x7fbfb80680b0 2023-10-06T04:32:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.966+0000 7fbfb67fc700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbfa4066f30 0x7fbfa40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.966+0000 7fbfb67fc700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbfa4066f30 0x7fbfa40693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:42.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:42.970+0000 7fbf9effd700 1 -- 172.21.15.123:0/3827903003 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fbfac0536d0 con 0x7fbfb80680b0 2023-10-06T04:32:43.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.147+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbfb80cde80 con 0x7fbfa4066f30 2023-10-06T04:32:43.159 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.157+0000 7fbf9effd700 1 -- 172.21.15.123:0/3827903003 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+391788 (secure 0 0 0) 0x7fbfb80cde80 con 0x7fbfa4066f30 2023-10-06T04:32:43.164 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:43.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbfa4066f30 msgr2=0x7fbfa40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:43.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbfa4066f30 0x7fbfa40693e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8006fd0 tx=0x7fbfa8008040 comp rx=0 tx=0).stop 2023-10-06T04:32:43.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80680b0 msgr2=0x7fbfb80c0a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:43.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80680b0 0x7fbfb80c0a10 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7fbfac0030c0 tx=0x7fbfac016000 comp rx=0 tx=0).stop 2023-10-06T04:32:43.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 shutdown_connections 2023-10-06T04:32:43.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fbfa4066f30 0x7fbfa40693e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:43.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fbfb80680b0 0x7fbfb80c0a10 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:43.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fbfb80ac720 0x7fbfb8156570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 --2- 172.21.15.123:0/3827903003 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fbfb806e250 0x7fbfb81591f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 >> 172.21.15.123:0/3827903003 conn(0x7fbfb80cda80 msgr2=0x7fbfb80c9850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 shutdown_connections 2023-10-06T04:32:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:43.161+0000 7fbfbd925700 1 -- 172.21.15.123:0/3827903003 wait complete. 2023-10-06T04:32:43.169 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:32:43.176 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":76,"stamp":"2023-10-06T04:32:42.902908+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395528,"kb_used_data":17224,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752579192,"statfs":{"total":773094113280,"available":770641092608,"internally_reserved":0,"allocated":17637376,"data_stored":15445072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.316608"},"pg_stats":[{"pgid":"6.3f","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.590115+0000","last_change":"2023-10-06T04:32:24.268479+0000","last_active":"2023-10-06T04:32:37.590115+0000","last_peered":"2023-10-06T04:32:37.590115+0000","last_clean":"2023-10-06T04:32:37.590115+0000","last_became_active":"2023-10-06T04:32:24.268175+0000","last_became_peered":"2023-10-06T04:32:24.268175+0000","last_unstale":"2023-10-06T04:32:37.590115+0000","last_undegraded":"2023-10-06T04:32:37.590115+0000","last_fullsized":"2023-10-06T04:32:37.590115+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"34'4","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589371+0000","last_change":"2023-10-06T04:32:24.269331+0000","last_active":"2023-10-06T04:32:37.589371+0000","last_peered":"2023-10-06T04:32:37.589371+0000","last_clean":"2023-10-06T04:32:37.589371+0000","last_became_active":"2023-10-06T04:32:24.269096+0000","last_became_peered":"2023-10-06T04:32:24.269096+0000","last_unstale":"2023-10-06T04:32:37.589371+0000","last_undegraded":"2023-10-06T04:32:37.589371+0000","last_fullsized":"2023-10-06T04:32:37.589371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592378+0000","last_change":"2023-10-06T04:32:24.267252+0000","last_active":"2023-10-06T04:32:37.592378+0000","last_peered":"2023-10-06T04:32:37.592378+0000","last_clean":"2023-10-06T04:32:37.592378+0000","last_became_active":"2023-10-06T04:32:24.266928+0000","last_became_peered":"2023-10-06T04:32:24.266928+0000","last_unstale":"2023-10-06T04:32:37.592378+0000","last_undegraded":"2023-10-06T04:32:37.592378+0000","last_fullsized":"2023-10-06T04:32:37.592378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"34'8","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593991+0000","last_change":"2023-10-06T04:32:24.271555+0000","last_active":"2023-10-06T04:32:37.593991+0000","last_peered":"2023-10-06T04:32:37.593991+0000","last_clean":"2023-10-06T04:32:37.593991+0000","last_became_active":"2023-10-06T04:32:24.271271+0000","last_became_peered":"2023-10-06T04:32:24.271271+0000","last_unstale":"2023-10-06T04:32:37.593991+0000","last_undegraded":"2023-10-06T04:32:37.593991+0000","last_fullsized":"2023-10-06T04:32:37.593991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"34'4","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592827+0000","last_change":"2023-10-06T04:32:24.270442+0000","last_active":"2023-10-06T04:32:37.592827+0000","last_peered":"2023-10-06T04:32:37.592827+0000","last_clean":"2023-10-06T04:32:37.592827+0000","last_became_active":"2023-10-06T04:32:24.270131+0000","last_became_peered":"2023-10-06T04:32:24.270131+0000","last_unstale":"2023-10-06T04:32:37.592827+0000","last_undegraded":"2023-10-06T04:32:37.592827+0000","last_fullsized":"2023-10-06T04:32:37.592827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596694+0000","last_change":"2023-10-06T04:32:24.263272+0000","last_active":"2023-10-06T04:32:37.596694+0000","last_peered":"2023-10-06T04:32:37.596694+0000","last_clean":"2023-10-06T04:32:37.596694+0000","last_became_active":"2023-10-06T04:32:24.262984+0000","last_became_peered":"2023-10-06T04:32:24.262984+0000","last_unstale":"2023-10-06T04:32:37.596694+0000","last_undegraded":"2023-10-06T04:32:37.596694+0000","last_fullsized":"2023-10-06T04:32:37.596694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:48.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"34'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596616+0000","last_change":"2023-10-06T04:32:24.263913+0000","last_active":"2023-10-06T04:32:37.596616+0000","last_peered":"2023-10-06T04:32:37.596616+0000","last_clean":"2023-10-06T04:32:37.596616+0000","last_became_active":"2023-10-06T04:32:24.263358+0000","last_became_peered":"2023-10-06T04:32:24.263358+0000","last_unstale":"2023-10-06T04:32:37.596616+0000","last_undegraded":"2023-10-06T04:32:37.596616+0000","last_fullsized":"2023-10-06T04:32:37.596616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589105+0000","last_change":"2023-10-06T04:32:24.266865+0000","last_active":"2023-10-06T04:32:37.589105+0000","last_peered":"2023-10-06T04:32:37.589105+0000","last_clean":"2023-10-06T04:32:37.589105+0000","last_became_active":"2023-10-06T04:32:24.266456+0000","last_became_peered":"2023-10-06T04:32:24.266456+0000","last_unstale":"2023-10-06T04:32:37.589105+0000","last_undegraded":"2023-10-06T04:32:37.589105+0000","last_fullsized":"2023-10-06T04:32:37.589105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"34'11","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592944+0000","last_change":"2023-10-06T04:32:24.267567+0000","last_active":"2023-10-06T04:32:37.592944+0000","last_peered":"2023-10-06T04:32:37.592944+0000","last_clean":"2023-10-06T04:32:37.592944+0000","last_became_active":"2023-10-06T04:32:24.267326+0000","last_became_peered":"2023-10-06T04:32:24.267326+0000","last_unstale":"2023-10-06T04:32:37.592944+0000","last_undegraded":"2023-10-06T04:32:37.592944+0000","last_fullsized":"2023-10-06T04:32:37.592944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"34'15","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598051+0000","last_change":"2023-10-06T04:32:24.266439+0000","last_active":"2023-10-06T04:32:37.598051+0000","last_peered":"2023-10-06T04:32:37.598051+0000","last_clean":"2023-10-06T04:32:37.598051+0000","last_became_active":"2023-10-06T04:32:24.266172+0000","last_became_peered":"2023-10-06T04:32:24.266172+0000","last_unstale":"2023-10-06T04:32:37.598051+0000","last_undegraded":"2023-10-06T04:32:37.598051+0000","last_fullsized":"2023-10-06T04:32:37.598051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:07.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"34'6","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597772+0000","last_change":"2023-10-06T04:32:24.264976+0000","last_active":"2023-10-06T04:32:37.597772+0000","last_peered":"2023-10-06T04:32:37.597772+0000","last_clean":"2023-10-06T04:32:37.597772+0000","last_became_active":"2023-10-06T04:32:24.264647+0000","last_became_peered":"2023-10-06T04:32:24.264647+0000","last_unstale":"2023-10-06T04:32:37.597772+0000","last_undegraded":"2023-10-06T04:32:37.597772+0000","last_fullsized":"2023-10-06T04:32:37.597772+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592448+0000","last_change":"2023-10-06T04:32:24.262121+0000","last_active":"2023-10-06T04:32:37.592448+0000","last_peered":"2023-10-06T04:32:37.592448+0000","last_clean":"2023-10-06T04:32:37.592448+0000","last_became_active":"2023-10-06T04:32:24.261846+0000","last_became_peered":"2023-10-06T04:32:24.261846+0000","last_unstale":"2023-10-06T04:32:37.592448+0000","last_undegraded":"2023-10-06T04:32:37.592448+0000","last_fullsized":"2023-10-06T04:32:37.592448+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"34'10","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597212+0000","last_change":"2023-10-06T04:32:24.267116+0000","last_active":"2023-10-06T04:32:37.597212+0000","last_peered":"2023-10-06T04:32:37.597212+0000","last_clean":"2023-10-06T04:32:37.597212+0000","last_became_active":"2023-10-06T04:32:24.266756+0000","last_became_peered":"2023-10-06T04:32:24.266756+0000","last_unstale":"2023-10-06T04:32:37.597212+0000","last_undegraded":"2023-10-06T04:32:37.597212+0000","last_fullsized":"2023-10-06T04:32:37.597212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:26.457033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601526+0000","last_change":"2023-10-06T04:32:24.267554+0000","last_active":"2023-10-06T04:32:37.601526+0000","last_peered":"2023-10-06T04:32:37.601526+0000","last_clean":"2023-10-06T04:32:37.601526+0000","last_became_active":"2023-10-06T04:32:24.266198+0000","last_became_peered":"2023-10-06T04:32:24.266198+0000","last_unstale":"2023-10-06T04:32:37.601526+0000","last_undegraded":"2023-10-06T04:32:37.601526+0000","last_fullsized":"2023-10-06T04:32:37.601526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"34'7","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595491+0000","last_change":"2023-10-06T04:32:24.265234+0000","last_active":"2023-10-06T04:32:37.595491+0000","last_peered":"2023-10-06T04:32:37.595491+0000","last_clean":"2023-10-06T04:32:37.595491+0000","last_became_active":"2023-10-06T04:32:24.264892+0000","last_became_peered":"2023-10-06T04:32:24.264892+0000","last_unstale":"2023-10-06T04:32:37.595491+0000","last_undegraded":"2023-10-06T04:32:37.595491+0000","last_fullsized":"2023-10-06T04:32:37.595491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595179+0000","last_change":"2023-10-06T04:32:24.268696+0000","last_active":"2023-10-06T04:32:37.595179+0000","last_peered":"2023-10-06T04:32:37.595179+0000","last_clean":"2023-10-06T04:32:37.595179+0000","last_became_active":"2023-10-06T04:32:24.268430+0000","last_became_peered":"2023-10-06T04:32:24.268430+0000","last_unstale":"2023-10-06T04:32:37.595179+0000","last_undegraded":"2023-10-06T04:32:37.595179+0000","last_fullsized":"2023-10-06T04:32:37.595179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595807+0000","last_change":"2023-10-06T04:32:24.269272+0000","last_active":"2023-10-06T04:32:37.595807+0000","last_peered":"2023-10-06T04:32:37.595807+0000","last_clean":"2023-10-06T04:32:37.595807+0000","last_became_active":"2023-10-06T04:32:24.268793+0000","last_became_peered":"2023-10-06T04:32:24.268793+0000","last_unstale":"2023-10-06T04:32:37.595807+0000","last_undegraded":"2023-10-06T04:32:37.595807+0000","last_fullsized":"2023-10-06T04:32:37.595807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"34'10","reported_seq":27,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595169+0000","last_change":"2023-10-06T04:32:24.268089+0000","last_active":"2023-10-06T04:32:37.595169+0000","last_peered":"2023-10-06T04:32:37.595169+0000","last_clean":"2023-10-06T04:32:37.595169+0000","last_became_active":"2023-10-06T04:32:24.267842+0000","last_became_peered":"2023-10-06T04:32:24.267842+0000","last_unstale":"2023-10-06T04:32:37.595169+0000","last_undegraded":"2023-10-06T04:32:37.595169+0000","last_fullsized":"2023-10-06T04:32:37.595169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:36.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"34'7","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592936+0000","last_change":"2023-10-06T04:32:24.263185+0000","last_active":"2023-10-06T04:32:37.592936+0000","last_peered":"2023-10-06T04:32:37.592936+0000","last_clean":"2023-10-06T04:32:37.592936+0000","last_became_active":"2023-10-06T04:32:24.262500+0000","last_became_peered":"2023-10-06T04:32:24.262500+0000","last_unstale":"2023-10-06T04:32:37.592936+0000","last_undegraded":"2023-10-06T04:32:37.592936+0000","last_fullsized":"2023-10-06T04:32:37.592936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:18.096304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"34'8","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594960+0000","last_change":"2023-10-06T04:32:24.266364+0000","last_active":"2023-10-06T04:32:37.594960+0000","last_peered":"2023-10-06T04:32:37.594960+0000","last_clean":"2023-10-06T04:32:37.594960+0000","last_became_active":"2023-10-06T04:32:24.266082+0000","last_became_peered":"2023-10-06T04:32:24.266082+0000","last_unstale":"2023-10-06T04:32:37.594960+0000","last_undegraded":"2023-10-06T04:32:37.594960+0000","last_fullsized":"2023-10-06T04:32:37.594960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"34'11","reported_seq":27,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597323+0000","last_change":"2023-10-06T04:32:24.265143+0000","last_active":"2023-10-06T04:32:37.597323+0000","last_peered":"2023-10-06T04:32:37.597323+0000","last_clean":"2023-10-06T04:32:37.597323+0000","last_became_active":"2023-10-06T04:32:24.264481+0000","last_became_peered":"2023-10-06T04:32:24.264481+0000","last_unstale":"2023-10-06T04:32:37.597323+0000","last_undegraded":"2023-10-06T04:32:37.597323+0000","last_fullsized":"2023-10-06T04:32:37.597323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:04.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601634+0000","last_change":"2023-10-06T04:32:24.262815+0000","last_active":"2023-10-06T04:32:37.601634+0000","last_peered":"2023-10-06T04:32:37.601634+0000","last_clean":"2023-10-06T04:32:37.601634+0000","last_became_active":"2023-10-06T04:32:24.262576+0000","last_became_peered":"2023-10-06T04:32:24.262576+0000","last_unstale":"2023-10-06T04:32:37.601634+0000","last_undegraded":"2023-10-06T04:32:37.601634+0000","last_fullsized":"2023-10-06T04:32:37.601634+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"34'8","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596537+0000","last_change":"2023-10-06T04:32:24.269201+0000","last_active":"2023-10-06T04:32:37.596537+0000","last_peered":"2023-10-06T04:32:37.596537+0000","last_clean":"2023-10-06T04:32:37.596537+0000","last_became_active":"2023-10-06T04:32:24.268841+0000","last_became_peered":"2023-10-06T04:32:24.268841+0000","last_unstale":"2023-10-06T04:32:37.596537+0000","last_undegraded":"2023-10-06T04:32:37.596537+0000","last_fullsized":"2023-10-06T04:32:37.596537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"34'6","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592840+0000","last_change":"2023-10-06T04:32:24.264875+0000","last_active":"2023-10-06T04:32:37.592840+0000","last_peered":"2023-10-06T04:32:37.592840+0000","last_clean":"2023-10-06T04:32:37.592840+0000","last_became_active":"2023-10-06T04:32:24.264601+0000","last_became_peered":"2023-10-06T04:32:24.264601+0000","last_unstale":"2023-10-06T04:32:37.592840+0000","last_undegraded":"2023-10-06T04:32:37.592840+0000","last_fullsized":"2023-10-06T04:32:37.592840+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:38.680988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"34'2","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595060+0000","last_change":"2023-10-06T04:32:24.268714+0000","last_active":"2023-10-06T04:32:37.595060+0000","last_peered":"2023-10-06T04:32:37.595060+0000","last_clean":"2023-10-06T04:32:37.595060+0000","last_became_active":"2023-10-06T04:32:24.268440+0000","last_became_peered":"2023-10-06T04:32:24.268440+0000","last_unstale":"2023-10-06T04:32:37.595060+0000","last_undegraded":"2023-10-06T04:32:37.595060+0000","last_fullsized":"2023-10-06T04:32:37.595060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:24.913639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"34'15","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595774+0000","last_change":"2023-10-06T04:32:24.266945+0000","last_active":"2023-10-06T04:32:37.595774+0000","last_peered":"2023-10-06T04:32:37.595774+0000","last_clean":"2023-10-06T04:32:37.595774+0000","last_became_active":"2023-10-06T04:32:24.266673+0000","last_became_peered":"2023-10-06T04:32:24.266673+0000","last_unstale":"2023-10-06T04:32:37.595774+0000","last_undegraded":"2023-10-06T04:32:37.595774+0000","last_fullsized":"2023-10-06T04:32:37.595774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"34'6","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595369+0000","last_change":"2023-10-06T04:32:24.270531+0000","last_active":"2023-10-06T04:32:37.595369+0000","last_peered":"2023-10-06T04:32:37.595369+0000","last_clean":"2023-10-06T04:32:37.595369+0000","last_became_active":"2023-10-06T04:32:24.270268+0000","last_became_peered":"2023-10-06T04:32:24.270268+0000","last_unstale":"2023-10-06T04:32:37.595369+0000","last_undegraded":"2023-10-06T04:32:37.595369+0000","last_fullsized":"2023-10-06T04:32:37.595369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:34.859497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"34'4","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597648+0000","last_change":"2023-10-06T04:32:24.265317+0000","last_active":"2023-10-06T04:32:37.597648+0000","last_peered":"2023-10-06T04:32:37.597648+0000","last_clean":"2023-10-06T04:32:37.597648+0000","last_became_active":"2023-10-06T04:32:24.264377+0000","last_became_peered":"2023-10-06T04:32:24.264377+0000","last_unstale":"2023-10-06T04:32:37.597648+0000","last_undegraded":"2023-10-06T04:32:37.597648+0000","last_fullsized":"2023-10-06T04:32:37.597648+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"34'7","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594958+0000","last_change":"2023-10-06T04:32:24.264442+0000","last_active":"2023-10-06T04:32:37.594958+0000","last_peered":"2023-10-06T04:32:37.594958+0000","last_clean":"2023-10-06T04:32:37.594958+0000","last_became_active":"2023-10-06T04:32:24.264148+0000","last_became_peered":"2023-10-06T04:32:24.264148+0000","last_unstale":"2023-10-06T04:32:37.594958+0000","last_undegraded":"2023-10-06T04:32:37.594958+0000","last_fullsized":"2023-10-06T04:32:37.594958+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"34'8","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594798+0000","last_change":"2023-10-06T04:32:24.270800+0000","last_active":"2023-10-06T04:32:37.594798+0000","last_peered":"2023-10-06T04:32:37.594798+0000","last_clean":"2023-10-06T04:32:37.594798+0000","last_became_active":"2023-10-06T04:32:24.270560+0000","last_became_peered":"2023-10-06T04:32:24.270560+0000","last_unstale":"2023-10-06T04:32:37.594798+0000","last_undegraded":"2023-10-06T04:32:37.594798+0000","last_fullsized":"2023-10-06T04:32:37.594798+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"34'12","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596416+0000","last_change":"2023-10-06T04:32:24.271609+0000","last_active":"2023-10-06T04:32:37.596416+0000","last_peered":"2023-10-06T04:32:37.596416+0000","last_clean":"2023-10-06T04:32:37.596416+0000","last_became_active":"2023-10-06T04:32:24.271354+0000","last_became_peered":"2023-10-06T04:32:24.271354+0000","last_unstale":"2023-10-06T04:32:37.596416+0000","last_undegraded":"2023-10-06T04:32:37.596416+0000","last_fullsized":"2023-10-06T04:32:37.596416+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594969+0000","last_change":"2023-10-06T04:32:24.263719+0000","last_active":"2023-10-06T04:32:37.594969+0000","last_peered":"2023-10-06T04:32:37.594969+0000","last_clean":"2023-10-06T04:32:37.594969+0000","last_became_active":"2023-10-06T04:32:24.263442+0000","last_became_peered":"2023-10-06T04:32:24.263442+0000","last_unstale":"2023-10-06T04:32:37.594969+0000","last_undegraded":"2023-10-06T04:32:37.594969+0000","last_fullsized":"2023-10-06T04:32:37.594969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:40.811547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.603644+0000","last_change":"2023-10-06T04:32:37.602186+0000","last_active":"2023-10-06T04:32:37.603644+0000","last_peered":"2023-10-06T04:32:37.603644+0000","last_clean":"2023-10-06T04:32:37.603644+0000","last_became_active":"2023-10-06T04:32:37.601966+0000","last_became_peered":"2023-10-06T04:32:37.601966+0000","last_unstale":"2023-10-06T04:32:37.603644+0000","last_undegraded":"2023-10-06T04:32:37.603644+0000","last_fullsized":"2023-10-06T04:32:37.603644+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:07.717408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593878+0000","last_change":"2023-10-06T04:32:24.270093+0000","last_active":"2023-10-06T04:32:37.593878+0000","last_peered":"2023-10-06T04:32:37.593878+0000","last_clean":"2023-10-06T04:32:37.593878+0000","last_became_active":"2023-10-06T04:32:24.269851+0000","last_became_peered":"2023-10-06T04:32:24.269851+0000","last_unstale":"2023-10-06T04:32:37.593878+0000","last_undegraded":"2023-10-06T04:32:37.593878+0000","last_fullsized":"2023-10-06T04:32:37.593878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'8","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595114+0000","last_change":"2023-10-06T04:32:23.441194+0000","last_active":"2023-10-06T04:32:37.595114+0000","last_peered":"2023-10-06T04:32:37.595114+0000","last_clean":"2023-10-06T04:32:37.595114+0000","last_became_active":"2023-10-06T04:32:23.440849+0000","last_became_peered":"2023-10-06T04:32:23.440849+0000","last_unstale":"2023-10-06T04:32:37.595114+0000","last_undegraded":"2023-10-06T04:32:37.595114+0000","last_fullsized":"2023-10-06T04:32:37.595114+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:59.829363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601055+0000","last_change":"2023-10-06T04:32:37.599756+0000","last_active":"2023-10-06T04:32:37.601055+0000","last_peered":"2023-10-06T04:32:37.601055+0000","last_clean":"2023-10-06T04:32:37.601055+0000","last_became_active":"2023-10-06T04:32:37.599479+0000","last_became_peered":"2023-10-06T04:32:37.599479+0000","last_unstale":"2023-10-06T04:32:37.601055+0000","last_undegraded":"2023-10-06T04:32:37.601055+0000","last_fullsized":"2023-10-06T04:32:37.601055+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:44.124162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"34'15","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598892+0000","last_change":"2023-10-06T04:32:24.267279+0000","last_active":"2023-10-06T04:32:37.598892+0000","last_peered":"2023-10-06T04:32:37.598892+0000","last_clean":"2023-10-06T04:32:37.598892+0000","last_became_active":"2023-10-06T04:32:24.267016+0000","last_became_peered":"2023-10-06T04:32:24.267016+0000","last_unstale":"2023-10-06T04:32:37.598892+0000","last_undegraded":"2023-10-06T04:32:37.598892+0000","last_fullsized":"2023-10-06T04:32:37.598892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593018+0000","last_change":"2023-10-06T04:32:23.450031+0000","last_active":"2023-10-06T04:32:37.593018+0000","last_peered":"2023-10-06T04:32:37.593018+0000","last_clean":"2023-10-06T04:32:37.593018+0000","last_became_active":"2023-10-06T04:32:23.449717+0000","last_became_peered":"2023-10-06T04:32:23.449717+0000","last_unstale":"2023-10-06T04:32:37.593018+0000","last_undegraded":"2023-10-06T04:32:37.593018+0000","last_fullsized":"2023-10-06T04:32:37.593018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:19.596388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605183+0000","last_change":"2023-10-06T04:32:37.604535+0000","last_active":"2023-10-06T04:32:37.605183+0000","last_peered":"2023-10-06T04:32:37.605183+0000","last_clean":"2023-10-06T04:32:37.605183+0000","last_became_active":"2023-10-06T04:32:37.604247+0000","last_became_peered":"2023-10-06T04:32:37.604247+0000","last_unstale":"2023-10-06T04:32:37.605183+0000","last_undegraded":"2023-10-06T04:32:37.605183+0000","last_fullsized":"2023-10-06T04:32:37.605183+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:39.617536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"34'5","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601119+0000","last_change":"2023-10-06T04:32:24.267302+0000","last_active":"2023-10-06T04:32:37.601119+0000","last_peered":"2023-10-06T04:32:37.601119+0000","last_clean":"2023-10-06T04:32:37.601119+0000","last_became_active":"2023-10-06T04:32:24.266976+0000","last_became_peered":"2023-10-06T04:32:24.266976+0000","last_unstale":"2023-10-06T04:32:37.601119+0000","last_undegraded":"2023-10-06T04:32:37.601119+0000","last_fullsized":"2023-10-06T04:32:37.601119+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598703+0000","last_change":"2023-10-06T04:32:23.447029+0000","last_active":"2023-10-06T04:32:37.598703+0000","last_peered":"2023-10-06T04:32:37.598703+0000","last_clean":"2023-10-06T04:32:37.598703+0000","last_became_active":"2023-10-06T04:32:23.446659+0000","last_became_peered":"2023-10-06T04:32:23.446659+0000","last_unstale":"2023-10-06T04:32:37.598703+0000","last_undegraded":"2023-10-06T04:32:37.598703+0000","last_fullsized":"2023-10-06T04:32:37.598703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:13.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600164+0000","last_change":"2023-10-06T04:32:37.598985+0000","last_active":"2023-10-06T04:32:37.600164+0000","last_peered":"2023-10-06T04:32:37.600164+0000","last_clean":"2023-10-06T04:32:37.600164+0000","last_became_active":"2023-10-06T04:32:37.598670+0000","last_became_peered":"2023-10-06T04:32:37.598670+0000","last_unstale":"2023-10-06T04:32:37.600164+0000","last_undegraded":"2023-10-06T04:32:37.600164+0000","last_fullsized":"2023-10-06T04:32:37.600164+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:52.813489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.590172+0000","last_change":"2023-10-06T04:32:24.265927+0000","last_active":"2023-10-06T04:32:37.590172+0000","last_peered":"2023-10-06T04:32:37.590172+0000","last_clean":"2023-10-06T04:32:37.590172+0000","last_became_active":"2023-10-06T04:32:24.265668+0000","last_became_peered":"2023-10-06T04:32:24.265668+0000","last_unstale":"2023-10-06T04:32:37.590172+0000","last_undegraded":"2023-10-06T04:32:37.590172+0000","last_fullsized":"2023-10-06T04:32:37.590172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600363+0000","last_change":"2023-10-06T04:32:23.437810+0000","last_active":"2023-10-06T04:32:37.600363+0000","last_peered":"2023-10-06T04:32:37.600363+0000","last_clean":"2023-10-06T04:32:37.600363+0000","last_became_active":"2023-10-06T04:32:23.437533+0000","last_became_peered":"2023-10-06T04:32:23.437533+0000","last_unstale":"2023-10-06T04:32:37.600363+0000","last_undegraded":"2023-10-06T04:32:37.600363+0000","last_fullsized":"2023-10-06T04:32:37.600363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:46.149228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604807+0000","last_change":"2023-10-06T04:32:37.600331+0000","last_active":"2023-10-06T04:32:37.604807+0000","last_peered":"2023-10-06T04:32:37.604807+0000","last_clean":"2023-10-06T04:32:37.604807+0000","last_became_active":"2023-10-06T04:32:37.600054+0000","last_became_peered":"2023-10-06T04:32:37.600054+0000","last_unstale":"2023-10-06T04:32:37.604807+0000","last_undegraded":"2023-10-06T04:32:37.604807+0000","last_fullsized":"2023-10-06T04:32:37.604807+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:37.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592055+0000","last_change":"2023-10-06T04:32:24.261934+0000","last_active":"2023-10-06T04:32:37.592055+0000","last_peered":"2023-10-06T04:32:37.592055+0000","last_clean":"2023-10-06T04:32:37.592055+0000","last_became_active":"2023-10-06T04:32:24.261681+0000","last_became_peered":"2023-10-06T04:32:24.261681+0000","last_unstale":"2023-10-06T04:32:37.592055+0000","last_undegraded":"2023-10-06T04:32:37.592055+0000","last_fullsized":"2023-10-06T04:32:37.592055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597480+0000","last_change":"2023-10-06T04:32:23.443656+0000","last_active":"2023-10-06T04:32:37.597480+0000","last_peered":"2023-10-06T04:32:37.597480+0000","last_clean":"2023-10-06T04:32:37.597480+0000","last_became_active":"2023-10-06T04:32:23.443293+0000","last_became_peered":"2023-10-06T04:32:23.443293+0000","last_unstale":"2023-10-06T04:32:37.597480+0000","last_undegraded":"2023-10-06T04:32:37.597480+0000","last_fullsized":"2023-10-06T04:32:37.597480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597831+0000","last_change":"2023-10-06T04:32:37.596919+0000","last_active":"2023-10-06T04:32:37.597831+0000","last_peered":"2023-10-06T04:32:37.597831+0000","last_clean":"2023-10-06T04:32:37.597831+0000","last_became_active":"2023-10-06T04:32:37.596678+0000","last_became_peered":"2023-10-06T04:32:37.596678+0000","last_unstale":"2023-10-06T04:32:37.597831+0000","last_undegraded":"2023-10-06T04:32:37.597831+0000","last_fullsized":"2023-10-06T04:32:37.597831+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:42.253389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"34'10","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596727+0000","last_change":"2023-10-06T04:32:24.264104+0000","last_active":"2023-10-06T04:32:37.596727+0000","last_peered":"2023-10-06T04:32:37.596727+0000","last_clean":"2023-10-06T04:32:37.596727+0000","last_became_active":"2023-10-06T04:32:24.263651+0000","last_became_peered":"2023-10-06T04:32:24.263651+0000","last_unstale":"2023-10-06T04:32:37.596727+0000","last_undegraded":"2023-10-06T04:32:37.596727+0000","last_fullsized":"2023-10-06T04:32:37.596727+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:24.395312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592700+0000","last_change":"2023-10-06T04:32:23.451767+0000","last_active":"2023-10-06T04:32:37.592700+0000","last_peered":"2023-10-06T04:32:37.592700+0000","last_clean":"2023-10-06T04:32:37.592700+0000","last_became_active":"2023-10-06T04:32:23.451430+0000","last_became_peered":"2023-10-06T04:32:23.451430+0000","last_unstale":"2023-10-06T04:32:37.592700+0000","last_undegraded":"2023-10-06T04:32:37.592700+0000","last_fullsized":"2023-10-06T04:32:37.592700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596676+0000","last_change":"2023-10-06T04:32:37.595151+0000","last_active":"2023-10-06T04:32:37.596676+0000","last_peered":"2023-10-06T04:32:37.596676+0000","last_clean":"2023-10-06T04:32:37.596676+0000","last_became_active":"2023-10-06T04:32:37.594826+0000","last_became_peered":"2023-10-06T04:32:37.594826+0000","last_unstale":"2023-10-06T04:32:37.596676+0000","last_undegraded":"2023-10-06T04:32:37.596676+0000","last_fullsized":"2023-10-06T04:32:37.596676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:01.505070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601760+0000","last_change":"2023-10-06T04:32:24.267750+0000","last_active":"2023-10-06T04:32:37.601760+0000","last_peered":"2023-10-06T04:32:37.601760+0000","last_clean":"2023-10-06T04:32:37.601760+0000","last_became_active":"2023-10-06T04:32:24.265805+0000","last_became_peered":"2023-10-06T04:32:24.265805+0000","last_unstale":"2023-10-06T04:32:37.601760+0000","last_undegraded":"2023-10-06T04:32:37.601760+0000","last_fullsized":"2023-10-06T04:32:37.601760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596297+0000","last_change":"2023-10-06T04:32:23.449278+0000","last_active":"2023-10-06T04:32:37.596297+0000","last_peered":"2023-10-06T04:32:37.596297+0000","last_clean":"2023-10-06T04:32:37.596297+0000","last_became_active":"2023-10-06T04:32:23.449034+0000","last_became_peered":"2023-10-06T04:32:23.449034+0000","last_unstale":"2023-10-06T04:32:37.596297+0000","last_undegraded":"2023-10-06T04:32:37.596297+0000","last_fullsized":"2023-10-06T04:32:37.596297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605423+0000","last_change":"2023-10-06T04:32:37.604424+0000","last_active":"2023-10-06T04:32:37.605423+0000","last_peered":"2023-10-06T04:32:37.605423+0000","last_clean":"2023-10-06T04:32:37.605423+0000","last_became_active":"2023-10-06T04:32:37.604099+0000","last_became_peered":"2023-10-06T04:32:37.604099+0000","last_unstale":"2023-10-06T04:32:37.605423+0000","last_undegraded":"2023-10-06T04:32:37.605423+0000","last_fullsized":"2023-10-06T04:32:37.605423+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:42.253389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589647+0000","last_change":"2023-10-06T04:32:24.265155+0000","last_active":"2023-10-06T04:32:37.589647+0000","last_peered":"2023-10-06T04:32:37.589647+0000","last_clean":"2023-10-06T04:32:37.589647+0000","last_became_active":"2023-10-06T04:32:24.264863+0000","last_became_peered":"2023-10-06T04:32:24.264863+0000","last_unstale":"2023-10-06T04:32:37.589647+0000","last_undegraded":"2023-10-06T04:32:37.589647+0000","last_fullsized":"2023-10-06T04:32:37.589647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.599649+0000","last_change":"2023-10-06T04:32:23.440295+0000","last_active":"2023-10-06T04:32:37.599649+0000","last_peered":"2023-10-06T04:32:37.599649+0000","last_clean":"2023-10-06T04:32:37.599649+0000","last_became_active":"2023-10-06T04:32:23.440039+0000","last_became_peered":"2023-10-06T04:32:23.440039+0000","last_unstale":"2023-10-06T04:32:37.599649+0000","last_undegraded":"2023-10-06T04:32:37.599649+0000","last_fullsized":"2023-10-06T04:32:37.599649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605123+0000","last_change":"2023-10-06T04:32:37.602775+0000","last_active":"2023-10-06T04:32:37.605123+0000","last_peered":"2023-10-06T04:32:37.605123+0000","last_clean":"2023-10-06T04:32:37.605123+0000","last_became_active":"2023-10-06T04:32:37.602538+0000","last_became_peered":"2023-10-06T04:32:37.602538+0000","last_unstale":"2023-10-06T04:32:37.605123+0000","last_undegraded":"2023-10-06T04:32:37.605123+0000","last_fullsized":"2023-10-06T04:32:37.605123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:08.107437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"34'11","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594091+0000","last_change":"2023-10-06T04:32:24.271227+0000","last_active":"2023-10-06T04:32:37.594091+0000","last_peered":"2023-10-06T04:32:37.594091+0000","last_clean":"2023-10-06T04:32:37.594091+0000","last_became_active":"2023-10-06T04:32:24.270984+0000","last_became_peered":"2023-10-06T04:32:24.270984+0000","last_unstale":"2023-10-06T04:32:37.594091+0000","last_undegraded":"2023-10-06T04:32:37.594091+0000","last_fullsized":"2023-10-06T04:32:37.594091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.590204+0000","last_change":"2023-10-06T04:32:23.447791+0000","last_active":"2023-10-06T04:32:37.590204+0000","last_peered":"2023-10-06T04:32:37.590204+0000","last_clean":"2023-10-06T04:32:37.590204+0000","last_became_active":"2023-10-06T04:32:23.447395+0000","last_became_peered":"2023-10-06T04:32:23.447395+0000","last_unstale":"2023-10-06T04:32:37.590204+0000","last_undegraded":"2023-10-06T04:32:37.590204+0000","last_fullsized":"2023-10-06T04:32:37.590204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:18.839316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598780+0000","last_change":"2023-10-06T04:32:37.597364+0000","last_active":"2023-10-06T04:32:37.598780+0000","last_peered":"2023-10-06T04:32:37.598780+0000","last_clean":"2023-10-06T04:32:37.598780+0000","last_became_active":"2023-10-06T04:32:37.597121+0000","last_became_peered":"2023-10-06T04:32:37.597121+0000","last_unstale":"2023-10-06T04:32:37.598780+0000","last_undegraded":"2023-10-06T04:32:37.598780+0000","last_fullsized":"2023-10-06T04:32:37.598780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:44.124162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"34'6","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593792+0000","last_change":"2023-10-06T04:32:24.269565+0000","last_active":"2023-10-06T04:32:37.593792+0000","last_peered":"2023-10-06T04:32:37.593792+0000","last_clean":"2023-10-06T04:32:37.593792+0000","last_became_active":"2023-10-06T04:32:24.269285+0000","last_became_peered":"2023-10-06T04:32:24.269285+0000","last_unstale":"2023-10-06T04:32:37.593792+0000","last_undegraded":"2023-10-06T04:32:37.593792+0000","last_fullsized":"2023-10-06T04:32:37.593792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598176+0000","last_change":"2023-10-06T04:32:23.444329+0000","last_active":"2023-10-06T04:32:37.598176+0000","last_peered":"2023-10-06T04:32:37.598176+0000","last_clean":"2023-10-06T04:32:37.598176+0000","last_became_active":"2023-10-06T04:32:23.444020+0000","last_became_peered":"2023-10-06T04:32:23.444020+0000","last_unstale":"2023-10-06T04:32:37.598176+0000","last_undegraded":"2023-10-06T04:32:37.598176+0000","last_fullsized":"2023-10-06T04:32:37.598176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605060+0000","last_change":"2023-10-06T04:32:37.601821+0000","last_active":"2023-10-06T04:32:37.605060+0000","last_peered":"2023-10-06T04:32:37.605060+0000","last_clean":"2023-10-06T04:32:37.605060+0000","last_became_active":"2023-10-06T04:32:37.601552+0000","last_became_peered":"2023-10-06T04:32:37.601552+0000","last_unstale":"2023-10-06T04:32:37.605060+0000","last_undegraded":"2023-10-06T04:32:37.605060+0000","last_fullsized":"2023-10-06T04:32:37.605060+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:45.220045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"34'10","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593007+0000","last_change":"2023-10-06T04:32:24.264060+0000","last_active":"2023-10-06T04:32:37.593007+0000","last_peered":"2023-10-06T04:32:37.593007+0000","last_clean":"2023-10-06T04:32:37.593007+0000","last_became_active":"2023-10-06T04:32:24.263825+0000","last_became_peered":"2023-10-06T04:32:24.263825+0000","last_unstale":"2023-10-06T04:32:37.593007+0000","last_undegraded":"2023-10-06T04:32:37.593007+0000","last_fullsized":"2023-10-06T04:32:37.593007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'5","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597797+0000","last_change":"2023-10-06T04:32:23.447887+0000","last_active":"2023-10-06T04:32:37.597797+0000","last_peered":"2023-10-06T04:32:37.597797+0000","last_clean":"2023-10-06T04:32:37.597797+0000","last_became_active":"2023-10-06T04:32:23.447650+0000","last_became_peered":"2023-10-06T04:32:23.447650+0000","last_unstale":"2023-10-06T04:32:37.597797+0000","last_undegraded":"2023-10-06T04:32:37.597797+0000","last_fullsized":"2023-10-06T04:32:37.597797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:47.164678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.607794+0000","last_change":"2023-10-06T04:32:37.605751+0000","last_active":"2023-10-06T04:32:37.607794+0000","last_peered":"2023-10-06T04:32:37.607794+0000","last_clean":"2023-10-06T04:32:37.607794+0000","last_became_active":"2023-10-06T04:32:37.605483+0000","last_became_peered":"2023-10-06T04:32:37.605483+0000","last_unstale":"2023-10-06T04:32:37.607794+0000","last_undegraded":"2023-10-06T04:32:37.607794+0000","last_fullsized":"2023-10-06T04:32:37.607794+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:44.627334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601573+0000","last_change":"2023-10-06T04:32:24.270050+0000","last_active":"2023-10-06T04:32:37.601573+0000","last_peered":"2023-10-06T04:32:37.601573+0000","last_clean":"2023-10-06T04:32:37.601573+0000","last_became_active":"2023-10-06T04:32:24.269298+0000","last_became_peered":"2023-10-06T04:32:24.269298+0000","last_unstale":"2023-10-06T04:32:37.601573+0000","last_undegraded":"2023-10-06T04:32:37.601573+0000","last_fullsized":"2023-10-06T04:32:37.601573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'22","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589493+0000","last_change":"2023-10-06T04:32:23.447643+0000","last_active":"2023-10-06T04:32:37.589493+0000","last_peered":"2023-10-06T04:32:37.589493+0000","last_clean":"2023-10-06T04:32:37.589493+0000","last_became_active":"2023-10-06T04:32:23.447352+0000","last_became_peered":"2023-10-06T04:32:23.447352+0000","last_unstale":"2023-10-06T04:32:37.589493+0000","last_undegraded":"2023-10-06T04:32:37.589493+0000","last_fullsized":"2023-10-06T04:32:37.589493+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605089+0000","last_change":"2023-10-06T04:32:37.601777+0000","last_active":"2023-10-06T04:32:37.605089+0000","last_peered":"2023-10-06T04:32:37.605089+0000","last_clean":"2023-10-06T04:32:37.605089+0000","last_became_active":"2023-10-06T04:32:37.601493+0000","last_became_peered":"2023-10-06T04:32:37.601493+0000","last_unstale":"2023-10-06T04:32:37.605089+0000","last_undegraded":"2023-10-06T04:32:37.605089+0000","last_fullsized":"2023-10-06T04:32:37.605089+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:09.297185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592167+0000","last_change":"2023-10-06T04:32:24.267086+0000","last_active":"2023-10-06T04:32:37.592167+0000","last_peered":"2023-10-06T04:32:37.592167+0000","last_clean":"2023-10-06T04:32:37.592167+0000","last_became_active":"2023-10-06T04:32:24.266829+0000","last_became_peered":"2023-10-06T04:32:24.266829+0000","last_unstale":"2023-10-06T04:32:37.592167+0000","last_undegraded":"2023-10-06T04:32:37.592167+0000","last_fullsized":"2023-10-06T04:32:37.592167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589881+0000","last_change":"2023-10-06T04:32:23.443222+0000","last_active":"2023-10-06T04:32:37.589881+0000","last_peered":"2023-10-06T04:32:37.589881+0000","last_clean":"2023-10-06T04:32:37.589881+0000","last_became_active":"2023-10-06T04:32:23.442957+0000","last_became_peered":"2023-10-06T04:32:23.442957+0000","last_unstale":"2023-10-06T04:32:37.589881+0000","last_undegraded":"2023-10-06T04:32:37.589881+0000","last_fullsized":"2023-10-06T04:32:37.589881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597828+0000","last_change":"2023-10-06T04:32:37.595162+0000","last_active":"2023-10-06T04:32:37.597828+0000","last_peered":"2023-10-06T04:32:37.597828+0000","last_clean":"2023-10-06T04:32:37.597828+0000","last_became_active":"2023-10-06T04:32:37.594902+0000","last_became_peered":"2023-10-06T04:32:37.594902+0000","last_unstale":"2023-10-06T04:32:37.597828+0000","last_undegraded":"2023-10-06T04:32:37.597828+0000","last_fullsized":"2023-10-06T04:32:37.597828+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:19.561294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589686+0000","last_change":"2023-10-06T04:32:24.268660+0000","last_active":"2023-10-06T04:32:37.589686+0000","last_peered":"2023-10-06T04:32:37.589686+0000","last_clean":"2023-10-06T04:32:37.589686+0000","last_became_active":"2023-10-06T04:32:24.268397+0000","last_became_peered":"2023-10-06T04:32:24.268397+0000","last_unstale":"2023-10-06T04:32:37.589686+0000","last_undegraded":"2023-10-06T04:32:37.589686+0000","last_fullsized":"2023-10-06T04:32:37.589686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596777+0000","last_change":"2023-10-06T04:32:23.445203+0000","last_active":"2023-10-06T04:32:37.596777+0000","last_peered":"2023-10-06T04:32:37.596777+0000","last_clean":"2023-10-06T04:32:37.596777+0000","last_became_active":"2023-10-06T04:32:23.444688+0000","last_became_peered":"2023-10-06T04:32:23.444688+0000","last_unstale":"2023-10-06T04:32:37.596777+0000","last_undegraded":"2023-10-06T04:32:37.596777+0000","last_fullsized":"2023-10-06T04:32:37.596777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600944+0000","last_change":"2023-10-06T04:32:37.599528+0000","last_active":"2023-10-06T04:32:37.600944+0000","last_peered":"2023-10-06T04:32:37.600944+0000","last_clean":"2023-10-06T04:32:37.600944+0000","last_became_active":"2023-10-06T04:32:37.599246+0000","last_became_peered":"2023-10-06T04:32:37.599246+0000","last_unstale":"2023-10-06T04:32:37.600944+0000","last_undegraded":"2023-10-06T04:32:37.600944+0000","last_fullsized":"2023-10-06T04:32:37.600944+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:07.670232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"34'10","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592580+0000","last_change":"2023-10-06T04:32:24.271729+0000","last_active":"2023-10-06T04:32:37.592580+0000","last_peered":"2023-10-06T04:32:37.592580+0000","last_clean":"2023-10-06T04:32:37.592580+0000","last_became_active":"2023-10-06T04:32:24.271453+0000","last_became_peered":"2023-10-06T04:32:24.271453+0000","last_unstale":"2023-10-06T04:32:37.592580+0000","last_undegraded":"2023-10-06T04:32:37.592580+0000","last_fullsized":"2023-10-06T04:32:37.592580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594348+0000","last_change":"2023-10-06T04:32:23.440530+0000","last_active":"2023-10-06T04:32:37.594348+0000","last_peered":"2023-10-06T04:32:37.594348+0000","last_clean":"2023-10-06T04:32:37.594348+0000","last_became_active":"2023-10-06T04:32:23.440226+0000","last_became_peered":"2023-10-06T04:32:23.440226+0000","last_unstale":"2023-10-06T04:32:37.594348+0000","last_undegraded":"2023-10-06T04:32:37.594348+0000","last_fullsized":"2023-10-06T04:32:37.594348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604942+0000","last_change":"2023-10-06T04:32:37.602515+0000","last_active":"2023-10-06T04:32:37.604942+0000","last_peered":"2023-10-06T04:32:37.604942+0000","last_clean":"2023-10-06T04:32:37.604942+0000","last_became_active":"2023-10-06T04:32:37.602239+0000","last_became_peered":"2023-10-06T04:32:37.602239+0000","last_unstale":"2023-10-06T04:32:37.604942+0000","last_undegraded":"2023-10-06T04:32:37.604942+0000","last_fullsized":"2023-10-06T04:32:37.604942+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:20.667728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"34'15","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589275+0000","last_change":"2023-10-06T04:32:24.268363+0000","last_active":"2023-10-06T04:32:37.589275+0000","last_peered":"2023-10-06T04:32:37.589275+0000","last_clean":"2023-10-06T04:32:37.589275+0000","last_became_active":"2023-10-06T04:32:24.268116+0000","last_became_peered":"2023-10-06T04:32:24.268116+0000","last_unstale":"2023-10-06T04:32:37.589275+0000","last_undegraded":"2023-10-06T04:32:37.589275+0000","last_fullsized":"2023-10-06T04:32:37.589275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'15","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595043+0000","last_change":"2023-10-06T04:32:23.452500+0000","last_active":"2023-10-06T04:32:37.595043+0000","last_peered":"2023-10-06T04:32:37.595043+0000","last_clean":"2023-10-06T04:32:37.595043+0000","last_became_active":"2023-10-06T04:32:23.452236+0000","last_became_peered":"2023-10-06T04:32:23.452236+0000","last_unstale":"2023-10-06T04:32:37.595043+0000","last_undegraded":"2023-10-06T04:32:37.595043+0000","last_fullsized":"2023-10-06T04:32:37.595043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604843+0000","last_change":"2023-10-06T04:32:37.601740+0000","last_active":"2023-10-06T04:32:37.604843+0000","last_peered":"2023-10-06T04:32:37.604843+0000","last_clean":"2023-10-06T04:32:37.604843+0000","last_became_active":"2023-10-06T04:32:37.600995+0000","last_became_peered":"2023-10-06T04:32:37.600995+0000","last_unstale":"2023-10-06T04:32:37.604843+0000","last_undegraded":"2023-10-06T04:32:37.604843+0000","last_fullsized":"2023-10-06T04:32:37.604843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:17.846635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589670+0000","last_change":"2023-10-06T04:32:24.265984+0000","last_active":"2023-10-06T04:32:37.589670+0000","last_peered":"2023-10-06T04:32:37.589670+0000","last_clean":"2023-10-06T04:32:37.589670+0000","last_became_active":"2023-10-06T04:32:24.265707+0000","last_became_peered":"2023-10-06T04:32:24.265707+0000","last_unstale":"2023-10-06T04:32:37.589670+0000","last_undegraded":"2023-10-06T04:32:37.589670+0000","last_fullsized":"2023-10-06T04:32:37.589670+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'2","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598844+0000","last_change":"2023-10-06T04:32:23.447020+0000","last_active":"2023-10-06T04:32:37.598844+0000","last_peered":"2023-10-06T04:32:37.598844+0000","last_clean":"2023-10-06T04:32:37.598844+0000","last_became_active":"2023-10-06T04:32:23.446786+0000","last_became_peered":"2023-10-06T04:32:23.446786+0000","last_unstale":"2023-10-06T04:32:37.598844+0000","last_undegraded":"2023-10-06T04:32:37.598844+0000","last_fullsized":"2023-10-06T04:32:37.598844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.805674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600795+0000","last_change":"2023-10-06T04:32:37.596846+0000","last_active":"2023-10-06T04:32:37.600795+0000","last_peered":"2023-10-06T04:32:37.600795+0000","last_clean":"2023-10-06T04:32:37.600795+0000","last_became_active":"2023-10-06T04:32:37.596333+0000","last_became_peered":"2023-10-06T04:32:37.596333+0000","last_unstale":"2023-10-06T04:32:37.600795+0000","last_undegraded":"2023-10-06T04:32:37.600795+0000","last_fullsized":"2023-10-06T04:32:37.600795+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:28.422241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"34'7","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597712+0000","last_change":"2023-10-06T04:32:24.265722+0000","last_active":"2023-10-06T04:32:37.597712+0000","last_peered":"2023-10-06T04:32:37.597712+0000","last_clean":"2023-10-06T04:32:37.597712+0000","last_became_active":"2023-10-06T04:32:24.265435+0000","last_became_peered":"2023-10-06T04:32:24.265435+0000","last_unstale":"2023-10-06T04:32:37.597712+0000","last_undegraded":"2023-10-06T04:32:37.597712+0000","last_fullsized":"2023-10-06T04:32:37.597712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.261738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601209+0000","last_change":"2023-10-06T04:32:23.442348+0000","last_active":"2023-10-06T04:32:37.601209+0000","last_peered":"2023-10-06T04:32:37.601209+0000","last_clean":"2023-10-06T04:32:37.601209+0000","last_became_active":"2023-10-06T04:32:23.441833+0000","last_became_peered":"2023-10-06T04:32:23.441833+0000","last_unstale":"2023-10-06T04:32:37.601209+0000","last_undegraded":"2023-10-06T04:32:37.601209+0000","last_fullsized":"2023-10-06T04:32:37.601209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.916159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600978+0000","last_change":"2023-10-06T04:32:37.598612+0000","last_active":"2023-10-06T04:32:37.600978+0000","last_peered":"2023-10-06T04:32:37.600978+0000","last_clean":"2023-10-06T04:32:37.600978+0000","last_became_active":"2023-10-06T04:32:37.598234+0000","last_became_peered":"2023-10-06T04:32:37.598234+0000","last_unstale":"2023-10-06T04:32:37.600978+0000","last_undegraded":"2023-10-06T04:32:37.600978+0000","last_fullsized":"2023-10-06T04:32:37.600978+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:09.297185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"34'5","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601001+0000","last_change":"2023-10-06T04:32:24.254622+0000","last_active":"2023-10-06T04:32:37.601001+0000","last_peered":"2023-10-06T04:32:37.601001+0000","last_clean":"2023-10-06T04:32:37.601001+0000","last_became_active":"2023-10-06T04:32:24.254294+0000","last_became_peered":"2023-10-06T04:32:24.254294+0000","last_unstale":"2023-10-06T04:32:37.601001+0000","last_undegraded":"2023-10-06T04:32:37.601001+0000","last_fullsized":"2023-10-06T04:32:37.601001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'2","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593025+0000","last_change":"2023-10-06T04:32:23.447777+0000","last_active":"2023-10-06T04:32:37.593025+0000","last_peered":"2023-10-06T04:32:37.593025+0000","last_clean":"2023-10-06T04:32:37.593025+0000","last_became_active":"2023-10-06T04:32:23.447526+0000","last_became_peered":"2023-10-06T04:32:23.447526+0000","last_unstale":"2023-10-06T04:32:37.593025+0000","last_undegraded":"2023-10-06T04:32:37.593025+0000","last_fullsized":"2023-10-06T04:32:37.593025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:59.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.602610+0000","last_change":"2023-10-06T04:32:37.600724+0000","last_active":"2023-10-06T04:32:37.602610+0000","last_peered":"2023-10-06T04:32:37.602610+0000","last_clean":"2023-10-06T04:32:37.602610+0000","last_became_active":"2023-10-06T04:32:37.600484+0000","last_became_peered":"2023-10-06T04:32:37.600484+0000","last_unstale":"2023-10-06T04:32:37.602610+0000","last_undegraded":"2023-10-06T04:32:37.602610+0000","last_fullsized":"2023-10-06T04:32:37.602610+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:09.231566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"34'9","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593221+0000","last_change":"2023-10-06T04:32:24.270632+0000","last_active":"2023-10-06T04:32:37.593221+0000","last_peered":"2023-10-06T04:32:37.593221+0000","last_clean":"2023-10-06T04:32:37.593221+0000","last_became_active":"2023-10-06T04:32:24.270263+0000","last_became_peered":"2023-10-06T04:32:24.270263+0000","last_unstale":"2023-10-06T04:32:37.593221+0000","last_undegraded":"2023-10-06T04:32:37.593221+0000","last_fullsized":"2023-10-06T04:32:37.593221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:19.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595065+0000","last_change":"2023-10-06T04:32:23.446504+0000","last_active":"2023-10-06T04:32:37.595065+0000","last_peered":"2023-10-06T04:32:37.595065+0000","last_clean":"2023-10-06T04:32:37.595065+0000","last_became_active":"2023-10-06T04:32:23.446221+0000","last_became_peered":"2023-10-06T04:32:23.446221+0000","last_unstale":"2023-10-06T04:32:37.595065+0000","last_undegraded":"2023-10-06T04:32:37.595065+0000","last_fullsized":"2023-10-06T04:32:37.595065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600085+0000","last_change":"2023-10-06T04:32:37.598125+0000","last_active":"2023-10-06T04:32:37.600085+0000","last_peered":"2023-10-06T04:32:37.600085+0000","last_clean":"2023-10-06T04:32:37.600085+0000","last_became_active":"2023-10-06T04:32:37.597871+0000","last_became_peered":"2023-10-06T04:32:37.597871+0000","last_unstale":"2023-10-06T04:32:37.600085+0000","last_undegraded":"2023-10-06T04:32:37.600085+0000","last_fullsized":"2023-10-06T04:32:37.600085+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:55.201937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595125+0000","last_change":"2023-10-06T04:32:24.271430+0000","last_active":"2023-10-06T04:32:37.595125+0000","last_peered":"2023-10-06T04:32:37.595125+0000","last_clean":"2023-10-06T04:32:37.595125+0000","last_became_active":"2023-10-06T04:32:24.271180+0000","last_became_peered":"2023-10-06T04:32:24.271180+0000","last_unstale":"2023-10-06T04:32:37.595125+0000","last_undegraded":"2023-10-06T04:32:37.595125+0000","last_fullsized":"2023-10-06T04:32:37.595125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'4","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593003+0000","last_change":"2023-10-06T04:32:23.445911+0000","last_active":"2023-10-06T04:32:37.593003+0000","last_peered":"2023-10-06T04:32:37.593003+0000","last_clean":"2023-10-06T04:32:37.593003+0000","last_became_active":"2023-10-06T04:32:23.445582+0000","last_became_peered":"2023-10-06T04:32:23.445582+0000","last_unstale":"2023-10-06T04:32:37.593003+0000","last_undegraded":"2023-10-06T04:32:37.593003+0000","last_fullsized":"2023-10-06T04:32:37.593003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:35.404671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600976+0000","last_change":"2023-10-06T04:32:37.600219+0000","last_active":"2023-10-06T04:32:37.600976+0000","last_peered":"2023-10-06T04:32:37.600976+0000","last_clean":"2023-10-06T04:32:37.600976+0000","last_became_active":"2023-10-06T04:32:37.599968+0000","last_became_peered":"2023-10-06T04:32:37.599968+0000","last_unstale":"2023-10-06T04:32:37.600976+0000","last_undegraded":"2023-10-06T04:32:37.600976+0000","last_fullsized":"2023-10-06T04:32:37.600976+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:38.127232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"34'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592552+0000","last_change":"2023-10-06T04:32:24.259250+0000","last_active":"2023-10-06T04:32:37.592552+0000","last_peered":"2023-10-06T04:32:37.592552+0000","last_clean":"2023-10-06T04:32:37.592552+0000","last_became_active":"2023-10-06T04:32:24.258962+0000","last_became_peered":"2023-10-06T04:32:24.258962+0000","last_unstale":"2023-10-06T04:32:37.592552+0000","last_undegraded":"2023-10-06T04:32:37.592552+0000","last_fullsized":"2023-10-06T04:32:37.592552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593227+0000","last_change":"2023-10-06T04:32:23.445529+0000","last_active":"2023-10-06T04:32:37.593227+0000","last_peered":"2023-10-06T04:32:37.593227+0000","last_clean":"2023-10-06T04:32:37.593227+0000","last_became_active":"2023-10-06T04:32:23.445245+0000","last_became_peered":"2023-10-06T04:32:23.445245+0000","last_unstale":"2023-10-06T04:32:37.593227+0000","last_undegraded":"2023-10-06T04:32:37.593227+0000","last_fullsized":"2023-10-06T04:32:37.593227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601271+0000","last_change":"2023-10-06T04:32:37.600779+0000","last_active":"2023-10-06T04:32:37.601271+0000","last_peered":"2023-10-06T04:32:37.601271+0000","last_clean":"2023-10-06T04:32:37.601271+0000","last_became_active":"2023-10-06T04:32:37.600484+0000","last_became_peered":"2023-10-06T04:32:37.600484+0000","last_unstale":"2023-10-06T04:32:37.601271+0000","last_undegraded":"2023-10-06T04:32:37.601271+0000","last_fullsized":"2023-10-06T04:32:37.601271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:31.160924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"34'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596902+0000","last_change":"2023-10-06T04:32:24.270658+0000","last_active":"2023-10-06T04:32:37.596902+0000","last_peered":"2023-10-06T04:32:37.596902+0000","last_clean":"2023-10-06T04:32:37.596902+0000","last_became_active":"2023-10-06T04:32:24.270394+0000","last_became_peered":"2023-10-06T04:32:24.270394+0000","last_unstale":"2023-10-06T04:32:37.596902+0000","last_undegraded":"2023-10-06T04:32:37.596902+0000","last_fullsized":"2023-10-06T04:32:37.596902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596641+0000","last_change":"2023-10-06T04:32:23.452178+0000","last_active":"2023-10-06T04:32:37.596641+0000","last_peered":"2023-10-06T04:32:37.596641+0000","last_clean":"2023-10-06T04:32:37.596641+0000","last_became_active":"2023-10-06T04:32:23.451849+0000","last_became_peered":"2023-10-06T04:32:23.451849+0000","last_unstale":"2023-10-06T04:32:37.596641+0000","last_undegraded":"2023-10-06T04:32:37.596641+0000","last_fullsized":"2023-10-06T04:32:37.596641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.026875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598585+0000","last_change":"2023-10-06T04:32:37.596334+0000","last_active":"2023-10-06T04:32:37.598585+0000","last_peered":"2023-10-06T04:32:37.598585+0000","last_clean":"2023-10-06T04:32:37.598585+0000","last_became_active":"2023-10-06T04:32:37.596085+0000","last_became_peered":"2023-10-06T04:32:37.596085+0000","last_unstale":"2023-10-06T04:32:37.598585+0000","last_undegraded":"2023-10-06T04:32:37.598585+0000","last_fullsized":"2023-10-06T04:32:37.598585+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:01.886035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589791+0000","last_change":"2023-10-06T04:32:24.265320+0000","last_active":"2023-10-06T04:32:37.589791+0000","last_peered":"2023-10-06T04:32:37.589791+0000","last_clean":"2023-10-06T04:32:37.589791+0000","last_became_active":"2023-10-06T04:32:24.264894+0000","last_became_peered":"2023-10-06T04:32:24.264894+0000","last_unstale":"2023-10-06T04:32:37.589791+0000","last_undegraded":"2023-10-06T04:32:37.589791+0000","last_fullsized":"2023-10-06T04:32:37.589791+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592944+0000","last_change":"2023-10-06T04:32:23.444136+0000","last_active":"2023-10-06T04:32:37.592944+0000","last_peered":"2023-10-06T04:32:37.592944+0000","last_clean":"2023-10-06T04:32:37.592944+0000","last_became_active":"2023-10-06T04:32:23.443839+0000","last_became_peered":"2023-10-06T04:32:23.443839+0000","last_unstale":"2023-10-06T04:32:37.592944+0000","last_undegraded":"2023-10-06T04:32:37.592944+0000","last_fullsized":"2023-10-06T04:32:37.592944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:10.814479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605032+0000","last_change":"2023-10-06T04:32:37.602480+0000","last_active":"2023-10-06T04:32:37.605032+0000","last_peered":"2023-10-06T04:32:37.605032+0000","last_clean":"2023-10-06T04:32:37.605032+0000","last_became_active":"2023-10-06T04:32:37.602207+0000","last_became_peered":"2023-10-06T04:32:37.602207+0000","last_unstale":"2023-10-06T04:32:37.605032+0000","last_undegraded":"2023-10-06T04:32:37.605032+0000","last_fullsized":"2023-10-06T04:32:37.605032+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:55.201937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592146+0000","last_change":"2023-10-06T04:32:24.269672+0000","last_active":"2023-10-06T04:32:37.592146+0000","last_peered":"2023-10-06T04:32:37.592146+0000","last_clean":"2023-10-06T04:32:37.592146+0000","last_became_active":"2023-10-06T04:32:24.269407+0000","last_became_peered":"2023-10-06T04:32:24.269407+0000","last_unstale":"2023-10-06T04:32:37.592146+0000","last_undegraded":"2023-10-06T04:32:37.592146+0000","last_fullsized":"2023-10-06T04:32:37.592146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592605+0000","last_change":"2023-10-06T04:32:23.446685+0000","last_active":"2023-10-06T04:32:37.592605+0000","last_peered":"2023-10-06T04:32:37.592605+0000","last_clean":"2023-10-06T04:32:37.592605+0000","last_became_active":"2023-10-06T04:32:23.446432+0000","last_became_peered":"2023-10-06T04:32:23.446432+0000","last_unstale":"2023-10-06T04:32:37.592605+0000","last_undegraded":"2023-10-06T04:32:37.592605+0000","last_fullsized":"2023-10-06T04:32:37.592605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595478+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:32:37.595478+0000","last_peered":"2023-10-06T04:32:37.595478+0000","last_clean":"2023-10-06T04:32:37.595478+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:32:37.595478+0000","last_undegraded":"2023-10-06T04:32:37.595478+0000","last_fullsized":"2023-10-06T04:32:37.595478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.603397+0000","last_change":"2023-10-06T04:32:37.599518+0000","last_active":"2023-10-06T04:32:37.603397+0000","last_peered":"2023-10-06T04:32:37.603397+0000","last_clean":"2023-10-06T04:32:37.603397+0000","last_became_active":"2023-10-06T04:32:37.598482+0000","last_became_peered":"2023-10-06T04:32:37.598482+0000","last_unstale":"2023-10-06T04:32:37.603397+0000","last_undegraded":"2023-10-06T04:32:37.603397+0000","last_fullsized":"2023-10-06T04:32:37.603397+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:14.898276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589246+0000","last_change":"2023-10-06T04:32:24.266084+0000","last_active":"2023-10-06T04:32:37.589246+0000","last_peered":"2023-10-06T04:32:37.589246+0000","last_clean":"2023-10-06T04:32:37.589246+0000","last_became_active":"2023-10-06T04:32:24.265847+0000","last_became_peered":"2023-10-06T04:32:24.265847+0000","last_unstale":"2023-10-06T04:32:37.589246+0000","last_undegraded":"2023-10-06T04:32:37.589246+0000","last_fullsized":"2023-10-06T04:32:37.589246+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593234+0000","last_change":"2023-10-06T04:32:23.451106+0000","last_active":"2023-10-06T04:32:37.593234+0000","last_peered":"2023-10-06T04:32:37.593234+0000","last_clean":"2023-10-06T04:32:37.593234+0000","last_became_active":"2023-10-06T04:32:23.450838+0000","last_became_peered":"2023-10-06T04:32:23.450838+0000","last_unstale":"2023-10-06T04:32:37.593234+0000","last_undegraded":"2023-10-06T04:32:37.593234+0000","last_fullsized":"2023-10-06T04:32:37.593234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597055+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:32:37.597055+0000","last_peered":"2023-10-06T04:32:37.597055+0000","last_clean":"2023-10-06T04:32:37.597055+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:32:37.597055+0000","last_undegraded":"2023-10-06T04:32:37.597055+0000","last_fullsized":"2023-10-06T04:32:37.597055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.607913+0000","last_change":"2023-10-06T04:32:37.604006+0000","last_active":"2023-10-06T04:32:37.607913+0000","last_peered":"2023-10-06T04:32:37.607913+0000","last_clean":"2023-10-06T04:32:37.607913+0000","last_became_active":"2023-10-06T04:32:37.603823+0000","last_became_peered":"2023-10-06T04:32:37.603823+0000","last_unstale":"2023-10-06T04:32:37.607913+0000","last_undegraded":"2023-10-06T04:32:37.607913+0000","last_fullsized":"2023-10-06T04:32:37.607913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:08.107437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"34'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593915+0000","last_change":"2023-10-06T04:32:24.272289+0000","last_active":"2023-10-06T04:32:37.593915+0000","last_peered":"2023-10-06T04:32:37.593915+0000","last_clean":"2023-10-06T04:32:37.593915+0000","last_became_active":"2023-10-06T04:32:24.271996+0000","last_became_peered":"2023-10-06T04:32:24.271996+0000","last_unstale":"2023-10-06T04:32:37.593915+0000","last_undegraded":"2023-10-06T04:32:37.593915+0000","last_fullsized":"2023-10-06T04:32:37.593915+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'3","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593048+0000","last_change":"2023-10-06T04:32:23.444286+0000","last_active":"2023-10-06T04:32:37.593048+0000","last_peered":"2023-10-06T04:32:37.593048+0000","last_clean":"2023-10-06T04:32:37.593048+0000","last_became_active":"2023-10-06T04:32:23.443982+0000","last_became_peered":"2023-10-06T04:32:23.443982+0000","last_unstale":"2023-10-06T04:32:37.593048+0000","last_undegraded":"2023-10-06T04:32:37.593048+0000","last_fullsized":"2023-10-06T04:32:37.593048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:20.394340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592892+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:32:37.592892+0000","last_peered":"2023-10-06T04:32:37.592892+0000","last_clean":"2023-10-06T04:32:37.592892+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:32:37.592892+0000","last_undegraded":"2023-10-06T04:32:37.592892+0000","last_fullsized":"2023-10-06T04:32:37.592892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601931+0000","last_change":"2023-10-06T04:32:37.600372+0000","last_active":"2023-10-06T04:32:37.601931+0000","last_peered":"2023-10-06T04:32:37.601931+0000","last_clean":"2023-10-06T04:32:37.601931+0000","last_became_active":"2023-10-06T04:32:37.600140+0000","last_became_peered":"2023-10-06T04:32:37.600140+0000","last_unstale":"2023-10-06T04:32:37.601931+0000","last_undegraded":"2023-10-06T04:32:37.601931+0000","last_fullsized":"2023-10-06T04:32:37.601931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:28.779614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"34'8","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598752+0000","last_change":"2023-10-06T04:32:24.265180+0000","last_active":"2023-10-06T04:32:37.598752+0000","last_peered":"2023-10-06T04:32:37.598752+0000","last_clean":"2023-10-06T04:32:37.598752+0000","last_became_active":"2023-10-06T04:32:24.264769+0000","last_became_peered":"2023-10-06T04:32:24.264769+0000","last_unstale":"2023-10-06T04:32:37.598752+0000","last_undegraded":"2023-10-06T04:32:37.598752+0000","last_fullsized":"2023-10-06T04:32:37.598752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593995+0000","last_change":"2023-10-06T04:32:23.452281+0000","last_active":"2023-10-06T04:32:37.593995+0000","last_peered":"2023-10-06T04:32:37.593995+0000","last_clean":"2023-10-06T04:32:37.593995+0000","last_became_active":"2023-10-06T04:32:23.452023+0000","last_became_peered":"2023-10-06T04:32:23.452023+0000","last_unstale":"2023-10-06T04:32:37.593995+0000","last_undegraded":"2023-10-06T04:32:37.593995+0000","last_fullsized":"2023-10-06T04:32:37.593995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597399+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:32:37.597399+0000","last_peered":"2023-10-06T04:32:37.597399+0000","last_clean":"2023-10-06T04:32:37.597399+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:32:37.597399+0000","last_undegraded":"2023-10-06T04:32:37.597399+0000","last_fullsized":"2023-10-06T04:32:37.597399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594956+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:32:37.594956+0000","last_peered":"2023-10-06T04:32:37.594956+0000","last_clean":"2023-10-06T04:32:37.594956+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:32:37.594956+0000","last_undegraded":"2023-10-06T04:32:37.594956+0000","last_fullsized":"2023-10-06T04:32:37.594956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600163+0000","last_change":"2023-10-06T04:32:37.599423+0000","last_active":"2023-10-06T04:32:37.600163+0000","last_peered":"2023-10-06T04:32:37.600163+0000","last_clean":"2023-10-06T04:32:37.600163+0000","last_became_active":"2023-10-06T04:32:37.599103+0000","last_became_peered":"2023-10-06T04:32:37.599103+0000","last_unstale":"2023-10-06T04:32:37.600163+0000","last_undegraded":"2023-10-06T04:32:37.600163+0000","last_fullsized":"2023-10-06T04:32:37.600163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:01.505070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594972+0000","last_change":"2023-10-06T04:32:24.267047+0000","last_active":"2023-10-06T04:32:37.594972+0000","last_peered":"2023-10-06T04:32:37.594972+0000","last_clean":"2023-10-06T04:32:37.594972+0000","last_became_active":"2023-10-06T04:32:24.266777+0000","last_became_peered":"2023-10-06T04:32:24.266777+0000","last_unstale":"2023-10-06T04:32:37.594972+0000","last_undegraded":"2023-10-06T04:32:37.594972+0000","last_fullsized":"2023-10-06T04:32:37.594972+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'3","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598932+0000","last_change":"2023-10-06T04:32:23.448480+0000","last_active":"2023-10-06T04:32:37.598932+0000","last_peered":"2023-10-06T04:32:37.598932+0000","last_clean":"2023-10-06T04:32:37.598932+0000","last_became_active":"2023-10-06T04:32:23.448242+0000","last_became_peered":"2023-10-06T04:32:23.448242+0000","last_unstale":"2023-10-06T04:32:37.598932+0000","last_undegraded":"2023-10-06T04:32:37.598932+0000","last_fullsized":"2023-10-06T04:32:37.598932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594875+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:32:37.594875+0000","last_peered":"2023-10-06T04:32:37.594875+0000","last_clean":"2023-10-06T04:32:37.594875+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:32:37.594875+0000","last_undegraded":"2023-10-06T04:32:37.594875+0000","last_fullsized":"2023-10-06T04:32:37.594875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604733+0000","last_change":"2023-10-06T04:32:37.602301+0000","last_active":"2023-10-06T04:32:37.604733+0000","last_peered":"2023-10-06T04:32:37.604733+0000","last_clean":"2023-10-06T04:32:37.604733+0000","last_became_active":"2023-10-06T04:32:37.601941+0000","last_became_peered":"2023-10-06T04:32:37.601941+0000","last_unstale":"2023-10-06T04:32:37.604733+0000","last_undegraded":"2023-10-06T04:32:37.604733+0000","last_fullsized":"2023-10-06T04:32:37.604733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:01.886035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"34'5","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596220+0000","last_change":"2023-10-06T04:32:24.266050+0000","last_active":"2023-10-06T04:32:37.596220+0000","last_peered":"2023-10-06T04:32:37.596220+0000","last_clean":"2023-10-06T04:32:37.596220+0000","last_became_active":"2023-10-06T04:32:24.265769+0000","last_became_peered":"2023-10-06T04:32:24.265769+0000","last_unstale":"2023-10-06T04:32:37.596220+0000","last_undegraded":"2023-10-06T04:32:37.596220+0000","last_fullsized":"2023-10-06T04:32:37.596220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:01.427293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594889+0000","last_change":"2023-10-06T04:32:23.443114+0000","last_active":"2023-10-06T04:32:37.594889+0000","last_peered":"2023-10-06T04:32:37.594889+0000","last_clean":"2023-10-06T04:32:37.594889+0000","last_became_active":"2023-10-06T04:32:23.442844+0000","last_became_peered":"2023-10-06T04:32:23.442844+0000","last_unstale":"2023-10-06T04:32:37.594889+0000","last_undegraded":"2023-10-06T04:32:37.594889+0000","last_fullsized":"2023-10-06T04:32:37.594889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594004+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:32:37.594004+0000","last_peered":"2023-10-06T04:32:37.594004+0000","last_clean":"2023-10-06T04:32:37.594004+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:32:37.594004+0000","last_undegraded":"2023-10-06T04:32:37.594004+0000","last_fullsized":"2023-10-06T04:32:37.594004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597853+0000","last_change":"2023-10-06T04:32:37.595103+0000","last_active":"2023-10-06T04:32:37.597853+0000","last_peered":"2023-10-06T04:32:37.597853+0000","last_clean":"2023-10-06T04:32:37.597853+0000","last_became_active":"2023-10-06T04:32:37.594835+0000","last_became_peered":"2023-10-06T04:32:37.594835+0000","last_unstale":"2023-10-06T04:32:37.597853+0000","last_undegraded":"2023-10-06T04:32:37.597853+0000","last_fullsized":"2023-10-06T04:32:37.597853+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:44.627334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601888+0000","last_change":"2023-10-06T04:32:24.265505+0000","last_active":"2023-10-06T04:32:37.601888+0000","last_peered":"2023-10-06T04:32:37.601888+0000","last_clean":"2023-10-06T04:32:37.601888+0000","last_became_active":"2023-10-06T04:32:24.265228+0000","last_became_peered":"2023-10-06T04:32:24.265228+0000","last_unstale":"2023-10-06T04:32:37.601888+0000","last_undegraded":"2023-10-06T04:32:37.601888+0000","last_fullsized":"2023-10-06T04:32:37.601888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600560+0000","last_change":"2023-10-06T04:32:23.438229+0000","last_active":"2023-10-06T04:32:37.600560+0000","last_peered":"2023-10-06T04:32:37.600560+0000","last_clean":"2023-10-06T04:32:37.600560+0000","last_became_active":"2023-10-06T04:32:23.437954+0000","last_became_peered":"2023-10-06T04:32:23.437954+0000","last_unstale":"2023-10-06T04:32:37.600560+0000","last_undegraded":"2023-10-06T04:32:37.600560+0000","last_fullsized":"2023-10-06T04:32:37.600560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:22.972008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595851+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:32:37.595851+0000","last_peered":"2023-10-06T04:32:37.595851+0000","last_clean":"2023-10-06T04:32:37.595851+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:32:37.595851+0000","last_undegraded":"2023-10-06T04:32:37.595851+0000","last_fullsized":"2023-10-06T04:32:37.595851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.607574+0000","last_change":"2023-10-06T04:32:37.601364+0000","last_active":"2023-10-06T04:32:37.607574+0000","last_peered":"2023-10-06T04:32:37.607574+0000","last_clean":"2023-10-06T04:32:37.607574+0000","last_became_active":"2023-10-06T04:32:37.600999+0000","last_became_peered":"2023-10-06T04:32:37.600999+0000","last_unstale":"2023-10-06T04:32:37.607574+0000","last_undegraded":"2023-10-06T04:32:37.607574+0000","last_fullsized":"2023-10-06T04:32:37.607574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:42.253389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"34'7","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593094+0000","last_change":"2023-10-06T04:32:24.272223+0000","last_active":"2023-10-06T04:32:37.593094+0000","last_peered":"2023-10-06T04:32:37.593094+0000","last_clean":"2023-10-06T04:32:37.593094+0000","last_became_active":"2023-10-06T04:32:24.271905+0000","last_became_peered":"2023-10-06T04:32:24.271905+0000","last_unstale":"2023-10-06T04:32:37.593094+0000","last_undegraded":"2023-10-06T04:32:37.593094+0000","last_fullsized":"2023-10-06T04:32:37.593094+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:17.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589921+0000","last_change":"2023-10-06T04:32:23.444321+0000","last_active":"2023-10-06T04:32:37.589921+0000","last_peered":"2023-10-06T04:32:37.589921+0000","last_clean":"2023-10-06T04:32:37.589921+0000","last_became_active":"2023-10-06T04:32:23.444073+0000","last_became_peered":"2023-10-06T04:32:23.444073+0000","last_unstale":"2023-10-06T04:32:37.589921+0000","last_undegraded":"2023-10-06T04:32:37.589921+0000","last_fullsized":"2023-10-06T04:32:37.589921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:37.675115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595148+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:32:37.595148+0000","last_peered":"2023-10-06T04:32:37.595148+0000","last_clean":"2023-10-06T04:32:37.595148+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:32:37.595148+0000","last_undegraded":"2023-10-06T04:32:37.595148+0000","last_fullsized":"2023-10-06T04:32:37.595148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300388,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071452,"statfs":{"total":96636764160,"available":96329166848,"internally_reserved":0,"allocated":3174400,"data_stored":2897698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.438},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.751}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.378},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.39400000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.311}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.42399999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.63800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298340,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073500,"statfs":{"total":96636764160,"available":96331264000,"internally_reserved":0,"allocated":1077248,"data_stored":800546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.34399999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.246}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.28499999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.374}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45100000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.441}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301856,"kb_used_data":4568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069984,"statfs":{"total":96636764160,"available":96327663616,"internally_reserved":0,"allocated":4677632,"data_stored":4404482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.375}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63700000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.68300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.46000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.373}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303904,"kb_used_data":6616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067936,"statfs":{"total":96636764160,"available":96325566464,"internally_reserved":0,"allocated":6774784,"data_stored":6501634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:32:46.185 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:32:46.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.318+0000 7f1b9508b700 1 Processor -- start 2023-10-06T04:32:46.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.318+0000 7f1b9508b700 1 -- start start 2023-10-06T04:32:46.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b9508b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1b901560c0 0x7f1b900c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b9508b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9006f080 0x7f1b9006e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b9508b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1b90156490 0x7f1b9006e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b9508b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900c8580 con 0x7f1b901560c0 2023-10-06T04:32:46.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b9508b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b900c8860 con 0x7f1b90156490 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b9508b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b900c8b40 con 0x7f1b9006f080 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1b901560c0 0x7f1b900c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1b901560c0 0x7f1b900c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33650/0 (socket says 172.21.15.123:33650) 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/169985483 learned_addr learned my addr 172.21.15.123:0/169985483 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8e59c700 1 --2- 172.21.15.123:0/169985483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9006f080 0x7f1b9006e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/169985483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9006f080 msgr2=0x7f1b9006e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8f59e700 1 --2- 172.21.15.123:0/169985483 >> v2:172.21.15.177:3300/0 conn(0x7f1b90156490 0x7f1b9006e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 --2- 172.21.15.123:0/169985483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9006f080 0x7f1b9006e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/169985483 >> v2:172.21.15.177:3300/0 conn(0x7f1b90156490 msgr2=0x7f1b9006e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:46.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 --2- 172.21.15.123:0/169985483 >> v2:172.21.15.177:3300/0 conn(0x7f1b90156490 0x7f1b9006e790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/169985483 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b900c8e20 con 0x7f1b901560c0 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8f59e700 1 --2- 172.21.15.123:0/169985483 >> v2:172.21.15.177:3300/0 conn(0x7f1b90156490 0x7f1b9006e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8e59c700 1 --2- 172.21.15.123:0/169985483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9006f080 0x7f1b9006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8ed9d700 1 --2- 172.21.15.123:0/169985483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b901560c0 0x7f1b900c7fa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1b900afb20 tx=0x7f1b80009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eb1feacb4c6e89e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8dd9b700 1 -- 172.21.15.123:0/169985483 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000b860 con 0x7f1b901560c0 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8dd9b700 1 -- 172.21.15.123:0/169985483 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b8000b9c0 con 0x7f1b901560c0 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.319+0000 7f1b8dd9b700 1 -- 172.21.15.123:0/169985483 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b8000bd60 con 0x7f1b901560c0 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 -- 172.21.15.123:0/169985483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b901560c0 msgr2=0x7f1b900c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 --2- 172.21.15.123:0/169985483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b901560c0 0x7f1b900c7fa0 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f1b900afb20 tx=0x7f1b80009250 comp rx=0 tx=0).stop 2023-10-06T04:32:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 -- 172.21.15.123:0/169985483 shutdown_connections 2023-10-06T04:32:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 --2- 172.21.15.123:0/169985483 >> v2:172.21.15.177:3300/0 conn(0x7f1b90156490 0x7f1b9006e790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 --2- 172.21.15.123:0/169985483 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b901560c0 0x7f1b900c7fa0 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 --2- 172.21.15.123:0/169985483 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b9006f080 0x7f1b9006e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 -- 172.21.15.123:0/169985483 >> 172.21.15.123:0/169985483 conn(0x7f1b900cda80 msgr2=0x7f1b900c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 -- 172.21.15.123:0/169985483 shutdown_connections 2023-10-06T04:32:46.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.320+0000 7f1b9508b700 1 -- 172.21.15.123:0/169985483 wait complete. 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.321+0000 7f1b9508b700 1 Processor -- start 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.321+0000 7f1b9508b700 1 -- start start 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b9508b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b9006f080 0x7f1b90152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b9508b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b901560c0 0x7f1b90158590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b9508b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b90156490 0x7f1b900ae880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b9508b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90159250 con 0x7f1b9006f080 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b9508b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b9015afd0 con 0x7f1b90156490 2023-10-06T04:32:46.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b9508b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b90158f70 con 0x7f1b901560c0 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b9006f080 0x7f1b90152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8ed9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b9006f080 0x7f1b90152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33662/0 (socket says 172.21.15.123:33662) 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/203238119 learned_addr learned my addr 172.21.15.123:0/203238119 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8f59e700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b90156490 0x7f1b900ae880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8e59c700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b901560c0 0x7f1b90158590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b901560c0 msgr2=0x7f1b90158590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8ed9d700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b901560c0 0x7f1b90158590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b90156490 msgr2=0x7f1b900ae880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.322+0000 7f1b8ed9d700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b90156490 0x7f1b900ae880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.323+0000 7f1b8ed9d700 1 -- 172.21.15.123:0/203238119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b800094f0 con 0x7f1b9006f080 2023-10-06T04:32:46.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.323+0000 7f1b8ed9d700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b9006f080 0x7f1b90152ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1b9006fd80 tx=0x7f1b80017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.323+0000 7f1b76ffd700 1 -- 172.21.15.123:0/203238119 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80024440 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.323+0000 7f1b76ffd700 1 -- 172.21.15.123:0/203238119 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b800245a0 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.323+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b9015a980 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.323+0000 7f1b76ffd700 1 -- 172.21.15.123:0/203238119 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b80024960 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.323+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b9015a630 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.324+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b900cde80 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.324+0000 7f1b76ffd700 1 -- 172.21.15.123:0/203238119 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b8001a420 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.325+0000 7f1b76ffd700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b7c066f60 0x7f1b7c069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.325+0000 7f1b76ffd700 1 -- 172.21.15.123:0/203238119 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f1b80086650 con 0x7f1b9006f080 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.325+0000 7f1b8e59c700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b7c066f60 0x7f1b7c069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.331 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.325+0000 7f1b8e59c700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b7c066f60 0x7f1b7c069410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400ab30 tx=0x7f1b84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:46.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.329+0000 7f1b76ffd700 1 -- 172.21.15.123:0/203238119 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b800535c0 con 0x7f1b9006f080 2023-10-06T04:32:46.510 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.508+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f1b9015b110 con 0x7f1b7c066f60 2023-10-06T04:32:46.522 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.518+0000 7f1b76ffd700 1 -- 172.21.15.123:0/203238119 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+391788 (secure 0 0 0) 0x7f1b8400f660 con 0x7f1b7c066f60 2023-10-06T04:32:46.526 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b7c066f60 msgr2=0x7f1b7c069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b7c066f60 0x7f1b7c069410 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7f1b8400ab30 tx=0x7f1b84008040 comp rx=0 tx=0).stop 2023-10-06T04:32:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b9006f080 msgr2=0x7f1b90152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.529 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b9006f080 0x7f1b90152ab0 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f1b9006fd80 tx=0x7f1b80017000 comp rx=0 tx=0).stop 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 shutdown_connections 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b7c066f60 0x7f1b7c069410 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b9006f080 0x7f1b90152ab0 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b90156490 0x7f1b900ae880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 --2- 172.21.15.123:0/203238119 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b901560c0 0x7f1b90158590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 >> 172.21.15.123:0/203238119 conn(0x7f1b900cda80 msgr2=0x7f1b900b1db0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 shutdown_connections 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.522+0000 7f1b9508b700 1 -- 172.21.15.123:0/203238119 wait complete. 2023-10-06T04:32:46.530 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:32:46.536 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":77,"stamp":"2023-10-06T04:32:44.903497+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":6881824,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":117,"num_read_kb":100,"num_write":255,"num_write_kb":8596,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":702,"ondisk_log_size":702,"up":274,"acting":274,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":274,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395528,"kb_used_data":17224,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752579192,"statfs":{"total":773094113280,"available":770641092608,"internally_reserved":0,"allocated":17637376,"data_stored":15445072,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":14,"apply_latency_ms":14,"commit_latency_ns":14000000,"apply_latency_ns":14000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001608"},"pg_stats":[{"pgid":"6.3f","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.590115+0000","last_change":"2023-10-06T04:32:24.268479+0000","last_active":"2023-10-06T04:32:37.590115+0000","last_peered":"2023-10-06T04:32:37.590115+0000","last_clean":"2023-10-06T04:32:37.590115+0000","last_became_active":"2023-10-06T04:32:24.268175+0000","last_became_peered":"2023-10-06T04:32:24.268175+0000","last_unstale":"2023-10-06T04:32:37.590115+0000","last_undegraded":"2023-10-06T04:32:37.590115+0000","last_fullsized":"2023-10-06T04:32:37.590115+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3e","version":"34'4","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589371+0000","last_change":"2023-10-06T04:32:24.269331+0000","last_active":"2023-10-06T04:32:37.589371+0000","last_peered":"2023-10-06T04:32:37.589371+0000","last_clean":"2023-10-06T04:32:37.589371+0000","last_became_active":"2023-10-06T04:32:24.269096+0000","last_became_peered":"2023-10-06T04:32:24.269096+0000","last_unstale":"2023-10-06T04:32:37.589371+0000","last_undegraded":"2023-10-06T04:32:37.589371+0000","last_fullsized":"2023-10-06T04:32:37.589371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3d","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592378+0000","last_change":"2023-10-06T04:32:24.267252+0000","last_active":"2023-10-06T04:32:37.592378+0000","last_peered":"2023-10-06T04:32:37.592378+0000","last_clean":"2023-10-06T04:32:37.592378+0000","last_became_active":"2023-10-06T04:32:24.266928+0000","last_became_peered":"2023-10-06T04:32:24.266928+0000","last_unstale":"2023-10-06T04:32:37.592378+0000","last_undegraded":"2023-10-06T04:32:37.592378+0000","last_fullsized":"2023-10-06T04:32:37.592378+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.3c","version":"34'8","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593991+0000","last_change":"2023-10-06T04:32:24.271555+0000","last_active":"2023-10-06T04:32:37.593991+0000","last_peered":"2023-10-06T04:32:37.593991+0000","last_clean":"2023-10-06T04:32:37.593991+0000","last_became_active":"2023-10-06T04:32:24.271271+0000","last_became_peered":"2023-10-06T04:32:24.271271+0000","last_unstale":"2023-10-06T04:32:37.593991+0000","last_undegraded":"2023-10-06T04:32:37.593991+0000","last_fullsized":"2023-10-06T04:32:37.593991+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3b","version":"34'4","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592827+0000","last_change":"2023-10-06T04:32:24.270442+0000","last_active":"2023-10-06T04:32:37.592827+0000","last_peered":"2023-10-06T04:32:37.592827+0000","last_clean":"2023-10-06T04:32:37.592827+0000","last_became_active":"2023-10-06T04:32:24.270131+0000","last_became_peered":"2023-10-06T04:32:24.270131+0000","last_unstale":"2023-10-06T04:32:37.592827+0000","last_undegraded":"2023-10-06T04:32:37.592827+0000","last_fullsized":"2023-10-06T04:32:37.592827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.3a","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596694+0000","last_change":"2023-10-06T04:32:24.263272+0000","last_active":"2023-10-06T04:32:37.596694+0000","last_peered":"2023-10-06T04:32:37.596694+0000","last_clean":"2023-10-06T04:32:37.596694+0000","last_became_active":"2023-10-06T04:32:24.262984+0000","last_became_peered":"2023-10-06T04:32:24.262984+0000","last_unstale":"2023-10-06T04:32:37.596694+0000","last_undegraded":"2023-10-06T04:32:37.596694+0000","last_fullsized":"2023-10-06T04:32:37.596694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:48.170552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.39","version":"34'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596616+0000","last_change":"2023-10-06T04:32:24.263913+0000","last_active":"2023-10-06T04:32:37.596616+0000","last_peered":"2023-10-06T04:32:37.596616+0000","last_clean":"2023-10-06T04:32:37.596616+0000","last_became_active":"2023-10-06T04:32:24.263358+0000","last_became_peered":"2023-10-06T04:32:24.263358+0000","last_unstale":"2023-10-06T04:32:37.596616+0000","last_undegraded":"2023-10-06T04:32:37.596616+0000","last_fullsized":"2023-10-06T04:32:37.596616+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.38","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589105+0000","last_change":"2023-10-06T04:32:24.266865+0000","last_active":"2023-10-06T04:32:37.589105+0000","last_peered":"2023-10-06T04:32:37.589105+0000","last_clean":"2023-10-06T04:32:37.589105+0000","last_became_active":"2023-10-06T04:32:24.266456+0000","last_became_peered":"2023-10-06T04:32:24.266456+0000","last_unstale":"2023-10-06T04:32:37.589105+0000","last_undegraded":"2023-10-06T04:32:37.589105+0000","last_fullsized":"2023-10-06T04:32:37.589105+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.37","version":"34'11","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592944+0000","last_change":"2023-10-06T04:32:24.267567+0000","last_active":"2023-10-06T04:32:37.592944+0000","last_peered":"2023-10-06T04:32:37.592944+0000","last_clean":"2023-10-06T04:32:37.592944+0000","last_became_active":"2023-10-06T04:32:24.267326+0000","last_became_peered":"2023-10-06T04:32:24.267326+0000","last_unstale":"2023-10-06T04:32:37.592944+0000","last_undegraded":"2023-10-06T04:32:37.592944+0000","last_fullsized":"2023-10-06T04:32:37.592944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.36","version":"34'15","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598051+0000","last_change":"2023-10-06T04:32:24.266439+0000","last_active":"2023-10-06T04:32:37.598051+0000","last_peered":"2023-10-06T04:32:37.598051+0000","last_clean":"2023-10-06T04:32:37.598051+0000","last_became_active":"2023-10-06T04:32:24.266172+0000","last_became_peered":"2023-10-06T04:32:24.266172+0000","last_unstale":"2023-10-06T04:32:37.598051+0000","last_undegraded":"2023-10-06T04:32:37.598051+0000","last_fullsized":"2023-10-06T04:32:37.598051+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:07.507225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.35","version":"34'6","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597772+0000","last_change":"2023-10-06T04:32:24.264976+0000","last_active":"2023-10-06T04:32:37.597772+0000","last_peered":"2023-10-06T04:32:37.597772+0000","last_clean":"2023-10-06T04:32:37.597772+0000","last_became_active":"2023-10-06T04:32:24.264647+0000","last_became_peered":"2023-10-06T04:32:24.264647+0000","last_unstale":"2023-10-06T04:32:37.597772+0000","last_undegraded":"2023-10-06T04:32:37.597772+0000","last_fullsized":"2023-10-06T04:32:37.597772+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.34","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592448+0000","last_change":"2023-10-06T04:32:24.262121+0000","last_active":"2023-10-06T04:32:37.592448+0000","last_peered":"2023-10-06T04:32:37.592448+0000","last_clean":"2023-10-06T04:32:37.592448+0000","last_became_active":"2023-10-06T04:32:24.261846+0000","last_became_peered":"2023-10-06T04:32:24.261846+0000","last_unstale":"2023-10-06T04:32:37.592448+0000","last_undegraded":"2023-10-06T04:32:37.592448+0000","last_fullsized":"2023-10-06T04:32:37.592448+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.33","version":"34'10","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597212+0000","last_change":"2023-10-06T04:32:24.267116+0000","last_active":"2023-10-06T04:32:37.597212+0000","last_peered":"2023-10-06T04:32:37.597212+0000","last_clean":"2023-10-06T04:32:37.597212+0000","last_became_active":"2023-10-06T04:32:24.266756+0000","last_became_peered":"2023-10-06T04:32:24.266756+0000","last_unstale":"2023-10-06T04:32:37.597212+0000","last_undegraded":"2023-10-06T04:32:37.597212+0000","last_fullsized":"2023-10-06T04:32:37.597212+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:26.457033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.32","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601526+0000","last_change":"2023-10-06T04:32:24.267554+0000","last_active":"2023-10-06T04:32:37.601526+0000","last_peered":"2023-10-06T04:32:37.601526+0000","last_clean":"2023-10-06T04:32:37.601526+0000","last_became_active":"2023-10-06T04:32:24.266198+0000","last_became_peered":"2023-10-06T04:32:24.266198+0000","last_unstale":"2023-10-06T04:32:37.601526+0000","last_undegraded":"2023-10-06T04:32:37.601526+0000","last_fullsized":"2023-10-06T04:32:37.601526+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.31","version":"34'7","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595491+0000","last_change":"2023-10-06T04:32:24.265234+0000","last_active":"2023-10-06T04:32:37.595491+0000","last_peered":"2023-10-06T04:32:37.595491+0000","last_clean":"2023-10-06T04:32:37.595491+0000","last_became_active":"2023-10-06T04:32:24.264892+0000","last_became_peered":"2023-10-06T04:32:24.264892+0000","last_unstale":"2023-10-06T04:32:37.595491+0000","last_undegraded":"2023-10-06T04:32:37.595491+0000","last_fullsized":"2023-10-06T04:32:37.595491+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.30","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595179+0000","last_change":"2023-10-06T04:32:24.268696+0000","last_active":"2023-10-06T04:32:37.595179+0000","last_peered":"2023-10-06T04:32:37.595179+0000","last_clean":"2023-10-06T04:32:37.595179+0000","last_became_active":"2023-10-06T04:32:24.268430+0000","last_became_peered":"2023-10-06T04:32:24.268430+0000","last_unstale":"2023-10-06T04:32:37.595179+0000","last_undegraded":"2023-10-06T04:32:37.595179+0000","last_fullsized":"2023-10-06T04:32:37.595179+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:03:30.107432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2f","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595807+0000","last_change":"2023-10-06T04:32:24.269272+0000","last_active":"2023-10-06T04:32:37.595807+0000","last_peered":"2023-10-06T04:32:37.595807+0000","last_clean":"2023-10-06T04:32:37.595807+0000","last_became_active":"2023-10-06T04:32:24.268793+0000","last_became_peered":"2023-10-06T04:32:24.268793+0000","last_unstale":"2023-10-06T04:32:37.595807+0000","last_undegraded":"2023-10-06T04:32:37.595807+0000","last_fullsized":"2023-10-06T04:32:37.595807+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2097152,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":2048,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.2e","version":"34'10","reported_seq":27,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595169+0000","last_change":"2023-10-06T04:32:24.268089+0000","last_active":"2023-10-06T04:32:37.595169+0000","last_peered":"2023-10-06T04:32:37.595169+0000","last_clean":"2023-10-06T04:32:37.595169+0000","last_became_active":"2023-10-06T04:32:24.267842+0000","last_became_peered":"2023-10-06T04:32:24.267842+0000","last_unstale":"2023-10-06T04:32:37.595169+0000","last_undegraded":"2023-10-06T04:32:37.595169+0000","last_fullsized":"2023-10-06T04:32:37.595169+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:36.947300+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2d","version":"34'7","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592936+0000","last_change":"2023-10-06T04:32:24.263185+0000","last_active":"2023-10-06T04:32:37.592936+0000","last_peered":"2023-10-06T04:32:37.592936+0000","last_clean":"2023-10-06T04:32:37.592936+0000","last_became_active":"2023-10-06T04:32:24.262500+0000","last_became_peered":"2023-10-06T04:32:24.262500+0000","last_unstale":"2023-10-06T04:32:37.592936+0000","last_undegraded":"2023-10-06T04:32:37.592936+0000","last_fullsized":"2023-10-06T04:32:37.592936+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:18.096304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.2c","version":"34'8","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594960+0000","last_change":"2023-10-06T04:32:24.266364+0000","last_active":"2023-10-06T04:32:37.594960+0000","last_peered":"2023-10-06T04:32:37.594960+0000","last_clean":"2023-10-06T04:32:37.594960+0000","last_became_active":"2023-10-06T04:32:24.266082+0000","last_became_peered":"2023-10-06T04:32:24.266082+0000","last_unstale":"2023-10-06T04:32:37.594960+0000","last_undegraded":"2023-10-06T04:32:37.594960+0000","last_fullsized":"2023-10-06T04:32:37.594960+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:36:21.361614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.2b","version":"34'11","reported_seq":27,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597323+0000","last_change":"2023-10-06T04:32:24.265143+0000","last_active":"2023-10-06T04:32:37.597323+0000","last_peered":"2023-10-06T04:32:37.597323+0000","last_clean":"2023-10-06T04:32:37.597323+0000","last_became_active":"2023-10-06T04:32:24.264481+0000","last_became_peered":"2023-10-06T04:32:24.264481+0000","last_unstale":"2023-10-06T04:32:37.597323+0000","last_undegraded":"2023-10-06T04:32:37.597323+0000","last_fullsized":"2023-10-06T04:32:37.597323+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:04.686132+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.2a","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601634+0000","last_change":"2023-10-06T04:32:24.262815+0000","last_active":"2023-10-06T04:32:37.601634+0000","last_peered":"2023-10-06T04:32:37.601634+0000","last_clean":"2023-10-06T04:32:37.601634+0000","last_became_active":"2023-10-06T04:32:24.262576+0000","last_became_peered":"2023-10-06T04:32:24.262576+0000","last_unstale":"2023-10-06T04:32:37.601634+0000","last_undegraded":"2023-10-06T04:32:37.601634+0000","last_fullsized":"2023-10-06T04:32:37.601634+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.29","version":"34'8","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596537+0000","last_change":"2023-10-06T04:32:24.269201+0000","last_active":"2023-10-06T04:32:37.596537+0000","last_peered":"2023-10-06T04:32:37.596537+0000","last_clean":"2023-10-06T04:32:37.596537+0000","last_became_active":"2023-10-06T04:32:24.268841+0000","last_became_peered":"2023-10-06T04:32:24.268841+0000","last_unstale":"2023-10-06T04:32:37.596537+0000","last_undegraded":"2023-10-06T04:32:37.596537+0000","last_fullsized":"2023-10-06T04:32:37.596537+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.28","version":"34'6","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592840+0000","last_change":"2023-10-06T04:32:24.264875+0000","last_active":"2023-10-06T04:32:37.592840+0000","last_peered":"2023-10-06T04:32:37.592840+0000","last_clean":"2023-10-06T04:32:37.592840+0000","last_became_active":"2023-10-06T04:32:24.264601+0000","last_became_peered":"2023-10-06T04:32:24.264601+0000","last_unstale":"2023-10-06T04:32:37.592840+0000","last_undegraded":"2023-10-06T04:32:37.592840+0000","last_fullsized":"2023-10-06T04:32:37.592840+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:38.680988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.27","version":"34'2","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595060+0000","last_change":"2023-10-06T04:32:24.268714+0000","last_active":"2023-10-06T04:32:37.595060+0000","last_peered":"2023-10-06T04:32:37.595060+0000","last_clean":"2023-10-06T04:32:37.595060+0000","last_became_active":"2023-10-06T04:32:24.268440+0000","last_became_peered":"2023-10-06T04:32:24.268440+0000","last_unstale":"2023-10-06T04:32:37.595060+0000","last_undegraded":"2023-10-06T04:32:37.595060+0000","last_fullsized":"2023-10-06T04:32:37.595060+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:24.913639+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.26","version":"34'15","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595774+0000","last_change":"2023-10-06T04:32:24.266945+0000","last_active":"2023-10-06T04:32:37.595774+0000","last_peered":"2023-10-06T04:32:37.595774+0000","last_clean":"2023-10-06T04:32:37.595774+0000","last_became_active":"2023-10-06T04:32:24.266673+0000","last_became_peered":"2023-10-06T04:32:24.266673+0000","last_unstale":"2023-10-06T04:32:37.595774+0000","last_undegraded":"2023-10-06T04:32:37.595774+0000","last_fullsized":"2023-10-06T04:32:37.595774+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:47.155101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.25","version":"34'6","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595369+0000","last_change":"2023-10-06T04:32:24.270531+0000","last_active":"2023-10-06T04:32:37.595369+0000","last_peered":"2023-10-06T04:32:37.595369+0000","last_clean":"2023-10-06T04:32:37.595369+0000","last_became_active":"2023-10-06T04:32:24.270268+0000","last_became_peered":"2023-10-06T04:32:24.270268+0000","last_unstale":"2023-10-06T04:32:37.595369+0000","last_undegraded":"2023-10-06T04:32:37.595369+0000","last_fullsized":"2023-10-06T04:32:37.595369+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:34.859497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.24","version":"34'4","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597648+0000","last_change":"2023-10-06T04:32:24.265317+0000","last_active":"2023-10-06T04:32:37.597648+0000","last_peered":"2023-10-06T04:32:37.597648+0000","last_clean":"2023-10-06T04:32:37.597648+0000","last_became_active":"2023-10-06T04:32:24.264377+0000","last_became_peered":"2023-10-06T04:32:24.264377+0000","last_unstale":"2023-10-06T04:32:37.597648+0000","last_undegraded":"2023-10-06T04:32:37.597648+0000","last_fullsized":"2023-10-06T04:32:37.597648+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.23","version":"34'7","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594958+0000","last_change":"2023-10-06T04:32:24.264442+0000","last_active":"2023-10-06T04:32:37.594958+0000","last_peered":"2023-10-06T04:32:37.594958+0000","last_clean":"2023-10-06T04:32:37.594958+0000","last_became_active":"2023-10-06T04:32:24.264148+0000","last_became_peered":"2023-10-06T04:32:24.264148+0000","last_unstale":"2023-10-06T04:32:37.594958+0000","last_undegraded":"2023-10-06T04:32:37.594958+0000","last_fullsized":"2023-10-06T04:32:37.594958+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:45:31.489088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.22","version":"34'8","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594798+0000","last_change":"2023-10-06T04:32:24.270800+0000","last_active":"2023-10-06T04:32:37.594798+0000","last_peered":"2023-10-06T04:32:37.594798+0000","last_clean":"2023-10-06T04:32:37.594798+0000","last_became_active":"2023-10-06T04:32:24.270560+0000","last_became_peered":"2023-10-06T04:32:24.270560+0000","last_unstale":"2023-10-06T04:32:37.594798+0000","last_undegraded":"2023-10-06T04:32:37.594798+0000","last_fullsized":"2023-10-06T04:32:37.594798+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.21","version":"34'12","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596416+0000","last_change":"2023-10-06T04:32:24.271609+0000","last_active":"2023-10-06T04:32:37.596416+0000","last_peered":"2023-10-06T04:32:37.596416+0000","last_clean":"2023-10-06T04:32:37.596416+0000","last_became_active":"2023-10-06T04:32:24.271354+0000","last_became_peered":"2023-10-06T04:32:24.271354+0000","last_unstale":"2023-10-06T04:32:37.596416+0000","last_undegraded":"2023-10-06T04:32:37.596416+0000","last_fullsized":"2023-10-06T04:32:37.596416+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.20","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594969+0000","last_change":"2023-10-06T04:32:24.263719+0000","last_active":"2023-10-06T04:32:37.594969+0000","last_peered":"2023-10-06T04:32:37.594969+0000","last_clean":"2023-10-06T04:32:37.594969+0000","last_became_active":"2023-10-06T04:32:24.263442+0000","last_became_peered":"2023-10-06T04:32:24.263442+0000","last_unstale":"2023-10-06T04:32:37.594969+0000","last_undegraded":"2023-10-06T04:32:37.594969+0000","last_fullsized":"2023-10-06T04:32:37.594969+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:40.811547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.603644+0000","last_change":"2023-10-06T04:32:37.602186+0000","last_active":"2023-10-06T04:32:37.603644+0000","last_peered":"2023-10-06T04:32:37.603644+0000","last_clean":"2023-10-06T04:32:37.603644+0000","last_became_active":"2023-10-06T04:32:37.601966+0000","last_became_peered":"2023-10-06T04:32:37.601966+0000","last_unstale":"2023-10-06T04:32:37.603644+0000","last_undegraded":"2023-10-06T04:32:37.603644+0000","last_fullsized":"2023-10-06T04:32:37.603644+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:07.717408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593878+0000","last_change":"2023-10-06T04:32:24.270093+0000","last_active":"2023-10-06T04:32:37.593878+0000","last_peered":"2023-10-06T04:32:37.593878+0000","last_clean":"2023-10-06T04:32:37.593878+0000","last_became_active":"2023-10-06T04:32:24.269851+0000","last_became_peered":"2023-10-06T04:32:24.269851+0000","last_unstale":"2023-10-06T04:32:37.593878+0000","last_undegraded":"2023-10-06T04:32:37.593878+0000","last_fullsized":"2023-10-06T04:32:37.593878+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1f","version":"35'8","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595114+0000","last_change":"2023-10-06T04:32:23.441194+0000","last_active":"2023-10-06T04:32:37.595114+0000","last_peered":"2023-10-06T04:32:37.595114+0000","last_clean":"2023-10-06T04:32:37.595114+0000","last_became_active":"2023-10-06T04:32:23.440849+0000","last_became_peered":"2023-10-06T04:32:23.440849+0000","last_unstale":"2023-10-06T04:32:37.595114+0000","last_undegraded":"2023-10-06T04:32:37.595114+0000","last_fullsized":"2023-10-06T04:32:37.595114+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:31:59.829363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":10,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601055+0000","last_change":"2023-10-06T04:32:37.599756+0000","last_active":"2023-10-06T04:32:37.601055+0000","last_peered":"2023-10-06T04:32:37.601055+0000","last_clean":"2023-10-06T04:32:37.601055+0000","last_became_active":"2023-10-06T04:32:37.599479+0000","last_became_peered":"2023-10-06T04:32:37.599479+0000","last_unstale":"2023-10-06T04:32:37.601055+0000","last_undegraded":"2023-10-06T04:32:37.601055+0000","last_fullsized":"2023-10-06T04:32:37.601055+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:44.124162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"34'15","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598892+0000","last_change":"2023-10-06T04:32:24.267279+0000","last_active":"2023-10-06T04:32:37.598892+0000","last_peered":"2023-10-06T04:32:37.598892+0000","last_clean":"2023-10-06T04:32:37.598892+0000","last_became_active":"2023-10-06T04:32:24.267016+0000","last_became_peered":"2023-10-06T04:32:24.267016+0000","last_unstale":"2023-10-06T04:32:37.598892+0000","last_undegraded":"2023-10-06T04:32:37.598892+0000","last_fullsized":"2023-10-06T04:32:37.598892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.1e","version":"33'1","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593018+0000","last_change":"2023-10-06T04:32:23.450031+0000","last_active":"2023-10-06T04:32:37.593018+0000","last_peered":"2023-10-06T04:32:37.593018+0000","last_clean":"2023-10-06T04:32:37.593018+0000","last_became_active":"2023-10-06T04:32:23.449717+0000","last_became_peered":"2023-10-06T04:32:23.449717+0000","last_unstale":"2023-10-06T04:32:37.593018+0000","last_undegraded":"2023-10-06T04:32:37.593018+0000","last_fullsized":"2023-10-06T04:32:37.593018+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:19.596388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605183+0000","last_change":"2023-10-06T04:32:37.604535+0000","last_active":"2023-10-06T04:32:37.605183+0000","last_peered":"2023-10-06T04:32:37.605183+0000","last_clean":"2023-10-06T04:32:37.605183+0000","last_became_active":"2023-10-06T04:32:37.604247+0000","last_became_peered":"2023-10-06T04:32:37.604247+0000","last_unstale":"2023-10-06T04:32:37.605183+0000","last_undegraded":"2023-10-06T04:32:37.605183+0000","last_fullsized":"2023-10-06T04:32:37.605183+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:53:39.617536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"34'5","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601119+0000","last_change":"2023-10-06T04:32:24.267302+0000","last_active":"2023-10-06T04:32:37.601119+0000","last_peered":"2023-10-06T04:32:37.601119+0000","last_clean":"2023-10-06T04:32:37.601119+0000","last_became_active":"2023-10-06T04:32:24.266976+0000","last_became_peered":"2023-10-06T04:32:24.266976+0000","last_unstale":"2023-10-06T04:32:37.601119+0000","last_undegraded":"2023-10-06T04:32:37.601119+0000","last_fullsized":"2023-10-06T04:32:37.601119+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1d","version":"35'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598703+0000","last_change":"2023-10-06T04:32:23.447029+0000","last_active":"2023-10-06T04:32:37.598703+0000","last_peered":"2023-10-06T04:32:37.598703+0000","last_clean":"2023-10-06T04:32:37.598703+0000","last_became_active":"2023-10-06T04:32:23.446659+0000","last_became_peered":"2023-10-06T04:32:23.446659+0000","last_unstale":"2023-10-06T04:32:37.598703+0000","last_undegraded":"2023-10-06T04:32:37.598703+0000","last_fullsized":"2023-10-06T04:32:37.598703+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:40:13.992750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600164+0000","last_change":"2023-10-06T04:32:37.598985+0000","last_active":"2023-10-06T04:32:37.600164+0000","last_peered":"2023-10-06T04:32:37.600164+0000","last_clean":"2023-10-06T04:32:37.600164+0000","last_became_active":"2023-10-06T04:32:37.598670+0000","last_became_peered":"2023-10-06T04:32:37.598670+0000","last_unstale":"2023-10-06T04:32:37.600164+0000","last_undegraded":"2023-10-06T04:32:37.600164+0000","last_fullsized":"2023-10-06T04:32:37.600164+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:06:52.813489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.590172+0000","last_change":"2023-10-06T04:32:24.265927+0000","last_active":"2023-10-06T04:32:37.590172+0000","last_peered":"2023-10-06T04:32:37.590172+0000","last_clean":"2023-10-06T04:32:37.590172+0000","last_became_active":"2023-10-06T04:32:24.265668+0000","last_became_peered":"2023-10-06T04:32:24.265668+0000","last_unstale":"2023-10-06T04:32:37.590172+0000","last_undegraded":"2023-10-06T04:32:37.590172+0000","last_fullsized":"2023-10-06T04:32:37.590172+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:25:36.897110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1c","version":"35'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600363+0000","last_change":"2023-10-06T04:32:23.437810+0000","last_active":"2023-10-06T04:32:37.600363+0000","last_peered":"2023-10-06T04:32:37.600363+0000","last_clean":"2023-10-06T04:32:37.600363+0000","last_became_active":"2023-10-06T04:32:23.437533+0000","last_became_peered":"2023-10-06T04:32:23.437533+0000","last_unstale":"2023-10-06T04:32:37.600363+0000","last_undegraded":"2023-10-06T04:32:37.600363+0000","last_fullsized":"2023-10-06T04:32:37.600363+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:46.149228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604807+0000","last_change":"2023-10-06T04:32:37.600331+0000","last_active":"2023-10-06T04:32:37.604807+0000","last_peered":"2023-10-06T04:32:37.604807+0000","last_clean":"2023-10-06T04:32:37.604807+0000","last_became_active":"2023-10-06T04:32:37.600054+0000","last_became_peered":"2023-10-06T04:32:37.600054+0000","last_unstale":"2023-10-06T04:32:37.604807+0000","last_undegraded":"2023-10-06T04:32:37.604807+0000","last_fullsized":"2023-10-06T04:32:37.604807+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:37.555815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592055+0000","last_change":"2023-10-06T04:32:24.261934+0000","last_active":"2023-10-06T04:32:37.592055+0000","last_peered":"2023-10-06T04:32:37.592055+0000","last_clean":"2023-10-06T04:32:37.592055+0000","last_became_active":"2023-10-06T04:32:24.261681+0000","last_became_peered":"2023-10-06T04:32:24.261681+0000","last_unstale":"2023-10-06T04:32:37.592055+0000","last_undegraded":"2023-10-06T04:32:37.592055+0000","last_fullsized":"2023-10-06T04:32:37.592055+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1b","version":"33'1","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597480+0000","last_change":"2023-10-06T04:32:23.443656+0000","last_active":"2023-10-06T04:32:37.597480+0000","last_peered":"2023-10-06T04:32:37.597480+0000","last_clean":"2023-10-06T04:32:37.597480+0000","last_became_active":"2023-10-06T04:32:23.443293+0000","last_became_peered":"2023-10-06T04:32:23.443293+0000","last_unstale":"2023-10-06T04:32:37.597480+0000","last_undegraded":"2023-10-06T04:32:37.597480+0000","last_fullsized":"2023-10-06T04:32:37.597480+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597831+0000","last_change":"2023-10-06T04:32:37.596919+0000","last_active":"2023-10-06T04:32:37.597831+0000","last_peered":"2023-10-06T04:32:37.597831+0000","last_clean":"2023-10-06T04:32:37.597831+0000","last_became_active":"2023-10-06T04:32:37.596678+0000","last_became_peered":"2023-10-06T04:32:37.596678+0000","last_unstale":"2023-10-06T04:32:37.597831+0000","last_undegraded":"2023-10-06T04:32:37.597831+0000","last_fullsized":"2023-10-06T04:32:37.597831+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:42.253389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"34'10","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596727+0000","last_change":"2023-10-06T04:32:24.264104+0000","last_active":"2023-10-06T04:32:37.596727+0000","last_peered":"2023-10-06T04:32:37.596727+0000","last_clean":"2023-10-06T04:32:37.596727+0000","last_became_active":"2023-10-06T04:32:24.263651+0000","last_became_peered":"2023-10-06T04:32:24.263651+0000","last_unstale":"2023-10-06T04:32:37.596727+0000","last_undegraded":"2023-10-06T04:32:37.596727+0000","last_fullsized":"2023-10-06T04:32:37.596727+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:57:24.395312+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.1a","version":"33'1","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592700+0000","last_change":"2023-10-06T04:32:23.451767+0000","last_active":"2023-10-06T04:32:37.592700+0000","last_peered":"2023-10-06T04:32:37.592700+0000","last_clean":"2023-10-06T04:32:37.592700+0000","last_became_active":"2023-10-06T04:32:23.451430+0000","last_became_peered":"2023-10-06T04:32:23.451430+0000","last_unstale":"2023-10-06T04:32:37.592700+0000","last_undegraded":"2023-10-06T04:32:37.592700+0000","last_fullsized":"2023-10-06T04:32:37.592700+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596676+0000","last_change":"2023-10-06T04:32:37.595151+0000","last_active":"2023-10-06T04:32:37.596676+0000","last_peered":"2023-10-06T04:32:37.596676+0000","last_clean":"2023-10-06T04:32:37.596676+0000","last_became_active":"2023-10-06T04:32:37.594826+0000","last_became_peered":"2023-10-06T04:32:37.594826+0000","last_unstale":"2023-10-06T04:32:37.596676+0000","last_undegraded":"2023-10-06T04:32:37.596676+0000","last_fullsized":"2023-10-06T04:32:37.596676+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:01.505070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601760+0000","last_change":"2023-10-06T04:32:24.267750+0000","last_active":"2023-10-06T04:32:37.601760+0000","last_peered":"2023-10-06T04:32:37.601760+0000","last_clean":"2023-10-06T04:32:37.601760+0000","last_became_active":"2023-10-06T04:32:24.265805+0000","last_became_peered":"2023-10-06T04:32:24.265805+0000","last_unstale":"2023-10-06T04:32:37.601760+0000","last_undegraded":"2023-10-06T04:32:37.601760+0000","last_fullsized":"2023-10-06T04:32:37.601760+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:58.357315+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596297+0000","last_change":"2023-10-06T04:32:23.449278+0000","last_active":"2023-10-06T04:32:37.596297+0000","last_peered":"2023-10-06T04:32:37.596297+0000","last_clean":"2023-10-06T04:32:37.596297+0000","last_became_active":"2023-10-06T04:32:23.449034+0000","last_became_peered":"2023-10-06T04:32:23.449034+0000","last_unstale":"2023-10-06T04:32:37.596297+0000","last_undegraded":"2023-10-06T04:32:37.596297+0000","last_fullsized":"2023-10-06T04:32:37.596297+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:47:23.907765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605423+0000","last_change":"2023-10-06T04:32:37.604424+0000","last_active":"2023-10-06T04:32:37.605423+0000","last_peered":"2023-10-06T04:32:37.605423+0000","last_clean":"2023-10-06T04:32:37.605423+0000","last_became_active":"2023-10-06T04:32:37.604099+0000","last_became_peered":"2023-10-06T04:32:37.604099+0000","last_unstale":"2023-10-06T04:32:37.605423+0000","last_undegraded":"2023-10-06T04:32:37.605423+0000","last_fullsized":"2023-10-06T04:32:37.605423+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:42.253389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589647+0000","last_change":"2023-10-06T04:32:24.265155+0000","last_active":"2023-10-06T04:32:37.589647+0000","last_peered":"2023-10-06T04:32:37.589647+0000","last_clean":"2023-10-06T04:32:37.589647+0000","last_became_active":"2023-10-06T04:32:24.264863+0000","last_became_peered":"2023-10-06T04:32:24.264863+0000","last_unstale":"2023-10-06T04:32:37.589647+0000","last_undegraded":"2023-10-06T04:32:37.589647+0000","last_fullsized":"2023-10-06T04:32:37.589647+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.599649+0000","last_change":"2023-10-06T04:32:23.440295+0000","last_active":"2023-10-06T04:32:37.599649+0000","last_peered":"2023-10-06T04:32:37.599649+0000","last_clean":"2023-10-06T04:32:37.599649+0000","last_became_active":"2023-10-06T04:32:23.440039+0000","last_became_peered":"2023-10-06T04:32:23.440039+0000","last_unstale":"2023-10-06T04:32:37.599649+0000","last_undegraded":"2023-10-06T04:32:37.599649+0000","last_fullsized":"2023-10-06T04:32:37.599649+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:00.421420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605123+0000","last_change":"2023-10-06T04:32:37.602775+0000","last_active":"2023-10-06T04:32:37.605123+0000","last_peered":"2023-10-06T04:32:37.605123+0000","last_clean":"2023-10-06T04:32:37.605123+0000","last_became_active":"2023-10-06T04:32:37.602538+0000","last_became_peered":"2023-10-06T04:32:37.602538+0000","last_unstale":"2023-10-06T04:32:37.605123+0000","last_undegraded":"2023-10-06T04:32:37.605123+0000","last_fullsized":"2023-10-06T04:32:37.605123+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:08.107437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"34'11","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594091+0000","last_change":"2023-10-06T04:32:24.271227+0000","last_active":"2023-10-06T04:32:37.594091+0000","last_peered":"2023-10-06T04:32:37.594091+0000","last_clean":"2023-10-06T04:32:37.594091+0000","last_became_active":"2023-10-06T04:32:24.270984+0000","last_became_peered":"2023-10-06T04:32:24.270984+0000","last_unstale":"2023-10-06T04:32:37.594091+0000","last_undegraded":"2023-10-06T04:32:37.594091+0000","last_fullsized":"2023-10-06T04:32:37.594091+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:39:20.602261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.590204+0000","last_change":"2023-10-06T04:32:23.447791+0000","last_active":"2023-10-06T04:32:37.590204+0000","last_peered":"2023-10-06T04:32:37.590204+0000","last_clean":"2023-10-06T04:32:37.590204+0000","last_became_active":"2023-10-06T04:32:23.447395+0000","last_became_peered":"2023-10-06T04:32:23.447395+0000","last_unstale":"2023-10-06T04:32:37.590204+0000","last_undegraded":"2023-10-06T04:32:37.590204+0000","last_fullsized":"2023-10-06T04:32:37.590204+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:18.839316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598780+0000","last_change":"2023-10-06T04:32:37.597364+0000","last_active":"2023-10-06T04:32:37.598780+0000","last_peered":"2023-10-06T04:32:37.598780+0000","last_clean":"2023-10-06T04:32:37.598780+0000","last_became_active":"2023-10-06T04:32:37.597121+0000","last_became_peered":"2023-10-06T04:32:37.597121+0000","last_unstale":"2023-10-06T04:32:37.598780+0000","last_undegraded":"2023-10-06T04:32:37.598780+0000","last_fullsized":"2023-10-06T04:32:37.598780+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:36:44.124162+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"34'6","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593792+0000","last_change":"2023-10-06T04:32:24.269565+0000","last_active":"2023-10-06T04:32:37.593792+0000","last_peered":"2023-10-06T04:32:37.593792+0000","last_clean":"2023-10-06T04:32:37.593792+0000","last_became_active":"2023-10-06T04:32:24.269285+0000","last_became_peered":"2023-10-06T04:32:24.269285+0000","last_unstale":"2023-10-06T04:32:37.593792+0000","last_undegraded":"2023-10-06T04:32:37.593792+0000","last_fullsized":"2023-10-06T04:32:37.593792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598176+0000","last_change":"2023-10-06T04:32:23.444329+0000","last_active":"2023-10-06T04:32:37.598176+0000","last_peered":"2023-10-06T04:32:37.598176+0000","last_clean":"2023-10-06T04:32:37.598176+0000","last_became_active":"2023-10-06T04:32:23.444020+0000","last_became_peered":"2023-10-06T04:32:23.444020+0000","last_unstale":"2023-10-06T04:32:37.598176+0000","last_undegraded":"2023-10-06T04:32:37.598176+0000","last_fullsized":"2023-10-06T04:32:37.598176+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605060+0000","last_change":"2023-10-06T04:32:37.601821+0000","last_active":"2023-10-06T04:32:37.605060+0000","last_peered":"2023-10-06T04:32:37.605060+0000","last_clean":"2023-10-06T04:32:37.605060+0000","last_became_active":"2023-10-06T04:32:37.601552+0000","last_became_peered":"2023-10-06T04:32:37.601552+0000","last_unstale":"2023-10-06T04:32:37.605060+0000","last_undegraded":"2023-10-06T04:32:37.605060+0000","last_fullsized":"2023-10-06T04:32:37.605060+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:13:45.220045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"34'10","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593007+0000","last_change":"2023-10-06T04:32:24.264060+0000","last_active":"2023-10-06T04:32:37.593007+0000","last_peered":"2023-10-06T04:32:37.593007+0000","last_clean":"2023-10-06T04:32:37.593007+0000","last_became_active":"2023-10-06T04:32:24.263825+0000","last_became_peered":"2023-10-06T04:32:24.263825+0000","last_unstale":"2023-10-06T04:32:37.593007+0000","last_undegraded":"2023-10-06T04:32:37.593007+0000","last_fullsized":"2023-10-06T04:32:37.593007+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:54:14.605307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.15","version":"35'5","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597797+0000","last_change":"2023-10-06T04:32:23.447887+0000","last_active":"2023-10-06T04:32:37.597797+0000","last_peered":"2023-10-06T04:32:37.597797+0000","last_clean":"2023-10-06T04:32:37.597797+0000","last_became_active":"2023-10-06T04:32:23.447650+0000","last_became_peered":"2023-10-06T04:32:23.447650+0000","last_unstale":"2023-10-06T04:32:37.597797+0000","last_undegraded":"2023-10-06T04:32:37.597797+0000","last_fullsized":"2023-10-06T04:32:37.597797+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:14:47.164678+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.607794+0000","last_change":"2023-10-06T04:32:37.605751+0000","last_active":"2023-10-06T04:32:37.607794+0000","last_peered":"2023-10-06T04:32:37.607794+0000","last_clean":"2023-10-06T04:32:37.607794+0000","last_became_active":"2023-10-06T04:32:37.605483+0000","last_became_peered":"2023-10-06T04:32:37.605483+0000","last_unstale":"2023-10-06T04:32:37.607794+0000","last_undegraded":"2023-10-06T04:32:37.607794+0000","last_fullsized":"2023-10-06T04:32:37.607794+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:44.627334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601573+0000","last_change":"2023-10-06T04:32:24.270050+0000","last_active":"2023-10-06T04:32:37.601573+0000","last_peered":"2023-10-06T04:32:37.601573+0000","last_clean":"2023-10-06T04:32:37.601573+0000","last_became_active":"2023-10-06T04:32:24.269298+0000","last_became_peered":"2023-10-06T04:32:24.269298+0000","last_unstale":"2023-10-06T04:32:37.601573+0000","last_undegraded":"2023-10-06T04:32:37.601573+0000","last_fullsized":"2023-10-06T04:32:37.601573+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.14","version":"35'22","reported_seq":38,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589493+0000","last_change":"2023-10-06T04:32:23.447643+0000","last_active":"2023-10-06T04:32:37.589493+0000","last_peered":"2023-10-06T04:32:37.589493+0000","last_clean":"2023-10-06T04:32:37.589493+0000","last_became_active":"2023-10-06T04:32:23.447352+0000","last_became_peered":"2023-10-06T04:32:23.447352+0000","last_unstale":"2023-10-06T04:32:37.589493+0000","last_undegraded":"2023-10-06T04:32:37.589493+0000","last_fullsized":"2023-10-06T04:32:37.589493+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":22,"log_dups_size":0,"ondisk_log_size":22,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":23,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605089+0000","last_change":"2023-10-06T04:32:37.601777+0000","last_active":"2023-10-06T04:32:37.605089+0000","last_peered":"2023-10-06T04:32:37.605089+0000","last_clean":"2023-10-06T04:32:37.605089+0000","last_became_active":"2023-10-06T04:32:37.601493+0000","last_became_peered":"2023-10-06T04:32:37.601493+0000","last_unstale":"2023-10-06T04:32:37.605089+0000","last_undegraded":"2023-10-06T04:32:37.605089+0000","last_fullsized":"2023-10-06T04:32:37.605089+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:09.297185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592167+0000","last_change":"2023-10-06T04:32:24.267086+0000","last_active":"2023-10-06T04:32:37.592167+0000","last_peered":"2023-10-06T04:32:37.592167+0000","last_clean":"2023-10-06T04:32:37.592167+0000","last_became_active":"2023-10-06T04:32:24.266829+0000","last_became_peered":"2023-10-06T04:32:24.266829+0000","last_unstale":"2023-10-06T04:32:37.592167+0000","last_undegraded":"2023-10-06T04:32:37.592167+0000","last_fullsized":"2023-10-06T04:32:37.592167+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:17:19.249869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.13","version":"35'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589881+0000","last_change":"2023-10-06T04:32:23.443222+0000","last_active":"2023-10-06T04:32:37.589881+0000","last_peered":"2023-10-06T04:32:37.589881+0000","last_clean":"2023-10-06T04:32:37.589881+0000","last_became_active":"2023-10-06T04:32:23.442957+0000","last_became_peered":"2023-10-06T04:32:23.442957+0000","last_unstale":"2023-10-06T04:32:37.589881+0000","last_undegraded":"2023-10-06T04:32:37.589881+0000","last_fullsized":"2023-10-06T04:32:37.589881+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:54:17.090430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597828+0000","last_change":"2023-10-06T04:32:37.595162+0000","last_active":"2023-10-06T04:32:37.597828+0000","last_peered":"2023-10-06T04:32:37.597828+0000","last_clean":"2023-10-06T04:32:37.597828+0000","last_became_active":"2023-10-06T04:32:37.594902+0000","last_became_peered":"2023-10-06T04:32:37.594902+0000","last_unstale":"2023-10-06T04:32:37.597828+0000","last_undegraded":"2023-10-06T04:32:37.597828+0000","last_fullsized":"2023-10-06T04:32:37.597828+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:39:19.561294+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589686+0000","last_change":"2023-10-06T04:32:24.268660+0000","last_active":"2023-10-06T04:32:37.589686+0000","last_peered":"2023-10-06T04:32:37.589686+0000","last_clean":"2023-10-06T04:32:37.589686+0000","last_became_active":"2023-10-06T04:32:24.268397+0000","last_became_peered":"2023-10-06T04:32:24.268397+0000","last_unstale":"2023-10-06T04:32:37.589686+0000","last_undegraded":"2023-10-06T04:32:37.589686+0000","last_fullsized":"2023-10-06T04:32:37.589686+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596777+0000","last_change":"2023-10-06T04:32:23.445203+0000","last_active":"2023-10-06T04:32:37.596777+0000","last_peered":"2023-10-06T04:32:37.596777+0000","last_clean":"2023-10-06T04:32:37.596777+0000","last_became_active":"2023-10-06T04:32:23.444688+0000","last_became_peered":"2023-10-06T04:32:23.444688+0000","last_unstale":"2023-10-06T04:32:37.596777+0000","last_undegraded":"2023-10-06T04:32:37.596777+0000","last_fullsized":"2023-10-06T04:32:37.596777+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600944+0000","last_change":"2023-10-06T04:32:37.599528+0000","last_active":"2023-10-06T04:32:37.600944+0000","last_peered":"2023-10-06T04:32:37.600944+0000","last_clean":"2023-10-06T04:32:37.600944+0000","last_became_active":"2023-10-06T04:32:37.599246+0000","last_became_peered":"2023-10-06T04:32:37.599246+0000","last_unstale":"2023-10-06T04:32:37.600944+0000","last_undegraded":"2023-10-06T04:32:37.600944+0000","last_fullsized":"2023-10-06T04:32:37.600944+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:35:07.670232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"34'10","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592580+0000","last_change":"2023-10-06T04:32:24.271729+0000","last_active":"2023-10-06T04:32:37.592580+0000","last_peered":"2023-10-06T04:32:37.592580+0000","last_clean":"2023-10-06T04:32:37.592580+0000","last_became_active":"2023-10-06T04:32:24.271453+0000","last_became_peered":"2023-10-06T04:32:24.271453+0000","last_unstale":"2023-10-06T04:32:37.592580+0000","last_undegraded":"2023-10-06T04:32:37.592580+0000","last_fullsized":"2023-10-06T04:32:37.592580+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:33:39.708340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594348+0000","last_change":"2023-10-06T04:32:23.440530+0000","last_active":"2023-10-06T04:32:37.594348+0000","last_peered":"2023-10-06T04:32:37.594348+0000","last_clean":"2023-10-06T04:32:37.594348+0000","last_became_active":"2023-10-06T04:32:23.440226+0000","last_became_peered":"2023-10-06T04:32:23.440226+0000","last_unstale":"2023-10-06T04:32:37.594348+0000","last_undegraded":"2023-10-06T04:32:37.594348+0000","last_fullsized":"2023-10-06T04:32:37.594348+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604942+0000","last_change":"2023-10-06T04:32:37.602515+0000","last_active":"2023-10-06T04:32:37.604942+0000","last_peered":"2023-10-06T04:32:37.604942+0000","last_clean":"2023-10-06T04:32:37.604942+0000","last_became_active":"2023-10-06T04:32:37.602239+0000","last_became_peered":"2023-10-06T04:32:37.602239+0000","last_unstale":"2023-10-06T04:32:37.604942+0000","last_undegraded":"2023-10-06T04:32:37.604942+0000","last_fullsized":"2023-10-06T04:32:37.604942+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:50:20.667728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"34'15","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589275+0000","last_change":"2023-10-06T04:32:24.268363+0000","last_active":"2023-10-06T04:32:37.589275+0000","last_peered":"2023-10-06T04:32:37.589275+0000","last_clean":"2023-10-06T04:32:37.589275+0000","last_became_active":"2023-10-06T04:32:24.268116+0000","last_became_peered":"2023-10-06T04:32:24.268116+0000","last_unstale":"2023-10-06T04:32:37.589275+0000","last_undegraded":"2023-10-06T04:32:37.589275+0000","last_fullsized":"2023-10-06T04:32:37.589275+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:55:10.162484+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.10","version":"35'15","reported_seq":32,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595043+0000","last_change":"2023-10-06T04:32:23.452500+0000","last_active":"2023-10-06T04:32:37.595043+0000","last_peered":"2023-10-06T04:32:37.595043+0000","last_clean":"2023-10-06T04:32:37.595043+0000","last_became_active":"2023-10-06T04:32:23.452236+0000","last_became_peered":"2023-10-06T04:32:23.452236+0000","last_unstale":"2023-10-06T04:32:37.595043+0000","last_undegraded":"2023-10-06T04:32:37.595043+0000","last_fullsized":"2023-10-06T04:32:37.595043+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":21,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604843+0000","last_change":"2023-10-06T04:32:37.601740+0000","last_active":"2023-10-06T04:32:37.604843+0000","last_peered":"2023-10-06T04:32:37.604843+0000","last_clean":"2023-10-06T04:32:37.604843+0000","last_became_active":"2023-10-06T04:32:37.600995+0000","last_became_peered":"2023-10-06T04:32:37.600995+0000","last_unstale":"2023-10-06T04:32:37.604843+0000","last_undegraded":"2023-10-06T04:32:37.604843+0000","last_fullsized":"2023-10-06T04:32:37.604843+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:57:17.846635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"34'6","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589670+0000","last_change":"2023-10-06T04:32:24.265984+0000","last_active":"2023-10-06T04:32:37.589670+0000","last_peered":"2023-10-06T04:32:37.589670+0000","last_clean":"2023-10-06T04:32:37.589670+0000","last_became_active":"2023-10-06T04:32:24.265707+0000","last_became_peered":"2023-10-06T04:32:24.265707+0000","last_unstale":"2023-10-06T04:32:37.589670+0000","last_undegraded":"2023-10-06T04:32:37.589670+0000","last_fullsized":"2023-10-06T04:32:37.589670+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.f","version":"35'2","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598844+0000","last_change":"2023-10-06T04:32:23.447020+0000","last_active":"2023-10-06T04:32:37.598844+0000","last_peered":"2023-10-06T04:32:37.598844+0000","last_clean":"2023-10-06T04:32:37.598844+0000","last_became_active":"2023-10-06T04:32:23.446786+0000","last_became_peered":"2023-10-06T04:32:23.446786+0000","last_unstale":"2023-10-06T04:32:37.598844+0000","last_undegraded":"2023-10-06T04:32:37.598844+0000","last_fullsized":"2023-10-06T04:32:37.598844+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:39.805674+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600795+0000","last_change":"2023-10-06T04:32:37.596846+0000","last_active":"2023-10-06T04:32:37.600795+0000","last_peered":"2023-10-06T04:32:37.600795+0000","last_clean":"2023-10-06T04:32:37.600795+0000","last_became_active":"2023-10-06T04:32:37.596333+0000","last_became_peered":"2023-10-06T04:32:37.596333+0000","last_unstale":"2023-10-06T04:32:37.600795+0000","last_undegraded":"2023-10-06T04:32:37.600795+0000","last_fullsized":"2023-10-06T04:32:37.600795+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:28.422241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"34'7","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597712+0000","last_change":"2023-10-06T04:32:24.265722+0000","last_active":"2023-10-06T04:32:37.597712+0000","last_peered":"2023-10-06T04:32:37.597712+0000","last_clean":"2023-10-06T04:32:37.597712+0000","last_became_active":"2023-10-06T04:32:24.265435+0000","last_became_peered":"2023-10-06T04:32:24.265435+0000","last_unstale":"2023-10-06T04:32:37.597712+0000","last_undegraded":"2023-10-06T04:32:37.597712+0000","last_fullsized":"2023-10-06T04:32:37.597712+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:00:15.261738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.e","version":"35'3","reported_seq":19,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601209+0000","last_change":"2023-10-06T04:32:23.442348+0000","last_active":"2023-10-06T04:32:37.601209+0000","last_peered":"2023-10-06T04:32:37.601209+0000","last_clean":"2023-10-06T04:32:37.601209+0000","last_became_active":"2023-10-06T04:32:23.441833+0000","last_became_peered":"2023-10-06T04:32:23.441833+0000","last_unstale":"2023-10-06T04:32:37.601209+0000","last_undegraded":"2023-10-06T04:32:37.601209+0000","last_fullsized":"2023-10-06T04:32:37.601209+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:21.916159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600978+0000","last_change":"2023-10-06T04:32:37.598612+0000","last_active":"2023-10-06T04:32:37.600978+0000","last_peered":"2023-10-06T04:32:37.600978+0000","last_clean":"2023-10-06T04:32:37.600978+0000","last_became_active":"2023-10-06T04:32:37.598234+0000","last_became_peered":"2023-10-06T04:32:37.598234+0000","last_unstale":"2023-10-06T04:32:37.600978+0000","last_undegraded":"2023-10-06T04:32:37.600978+0000","last_fullsized":"2023-10-06T04:32:37.600978+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:51:09.297185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"34'5","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601001+0000","last_change":"2023-10-06T04:32:24.254622+0000","last_active":"2023-10-06T04:32:37.601001+0000","last_peered":"2023-10-06T04:32:37.601001+0000","last_clean":"2023-10-06T04:32:37.601001+0000","last_became_active":"2023-10-06T04:32:24.254294+0000","last_became_peered":"2023-10-06T04:32:24.254294+0000","last_unstale":"2023-10-06T04:32:37.601001+0000","last_undegraded":"2023-10-06T04:32:37.601001+0000","last_fullsized":"2023-10-06T04:32:37.601001+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:01.737773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"35'2","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593025+0000","last_change":"2023-10-06T04:32:23.447777+0000","last_active":"2023-10-06T04:32:37.593025+0000","last_peered":"2023-10-06T04:32:37.593025+0000","last_clean":"2023-10-06T04:32:37.593025+0000","last_became_active":"2023-10-06T04:32:23.447526+0000","last_became_peered":"2023-10-06T04:32:23.447526+0000","last_unstale":"2023-10-06T04:32:37.593025+0000","last_undegraded":"2023-10-06T04:32:37.593025+0000","last_fullsized":"2023-10-06T04:32:37.593025+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:09:59.811767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.602610+0000","last_change":"2023-10-06T04:32:37.600724+0000","last_active":"2023-10-06T04:32:37.602610+0000","last_peered":"2023-10-06T04:32:37.602610+0000","last_clean":"2023-10-06T04:32:37.602610+0000","last_became_active":"2023-10-06T04:32:37.600484+0000","last_became_peered":"2023-10-06T04:32:37.600484+0000","last_unstale":"2023-10-06T04:32:37.602610+0000","last_undegraded":"2023-10-06T04:32:37.602610+0000","last_fullsized":"2023-10-06T04:32:37.602610+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:36:09.231566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"34'9","reported_seq":26,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593221+0000","last_change":"2023-10-06T04:32:24.270632+0000","last_active":"2023-10-06T04:32:37.593221+0000","last_peered":"2023-10-06T04:32:37.593221+0000","last_clean":"2023-10-06T04:32:37.593221+0000","last_became_active":"2023-10-06T04:32:24.270263+0000","last_became_peered":"2023-10-06T04:32:24.270263+0000","last_unstale":"2023-10-06T04:32:37.593221+0000","last_undegraded":"2023-10-06T04:32:37.593221+0000","last_fullsized":"2023-10-06T04:32:37.593221+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:36:19.845190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595065+0000","last_change":"2023-10-06T04:32:23.446504+0000","last_active":"2023-10-06T04:32:37.595065+0000","last_peered":"2023-10-06T04:32:37.595065+0000","last_clean":"2023-10-06T04:32:37.595065+0000","last_became_active":"2023-10-06T04:32:23.446221+0000","last_became_peered":"2023-10-06T04:32:23.446221+0000","last_unstale":"2023-10-06T04:32:37.595065+0000","last_undegraded":"2023-10-06T04:32:37.595065+0000","last_fullsized":"2023-10-06T04:32:37.595065+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600085+0000","last_change":"2023-10-06T04:32:37.598125+0000","last_active":"2023-10-06T04:32:37.600085+0000","last_peered":"2023-10-06T04:32:37.600085+0000","last_clean":"2023-10-06T04:32:37.600085+0000","last_became_active":"2023-10-06T04:32:37.597871+0000","last_became_peered":"2023-10-06T04:32:37.597871+0000","last_unstale":"2023-10-06T04:32:37.600085+0000","last_undegraded":"2023-10-06T04:32:37.600085+0000","last_fullsized":"2023-10-06T04:32:37.600085+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:55.201937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"34'9","reported_seq":25,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595125+0000","last_change":"2023-10-06T04:32:24.271430+0000","last_active":"2023-10-06T04:32:37.595125+0000","last_peered":"2023-10-06T04:32:37.595125+0000","last_clean":"2023-10-06T04:32:37.595125+0000","last_became_active":"2023-10-06T04:32:24.271180+0000","last_became_peered":"2023-10-06T04:32:24.271180+0000","last_unstale":"2023-10-06T04:32:37.595125+0000","last_undegraded":"2023-10-06T04:32:37.595125+0000","last_fullsized":"2023-10-06T04:32:37.595125+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:50:22.922033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.b","version":"35'4","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593003+0000","last_change":"2023-10-06T04:32:23.445911+0000","last_active":"2023-10-06T04:32:37.593003+0000","last_peered":"2023-10-06T04:32:37.593003+0000","last_clean":"2023-10-06T04:32:37.593003+0000","last_became_active":"2023-10-06T04:32:23.445582+0000","last_became_peered":"2023-10-06T04:32:23.445582+0000","last_unstale":"2023-10-06T04:32:37.593003+0000","last_undegraded":"2023-10-06T04:32:37.593003+0000","last_fullsized":"2023-10-06T04:32:37.593003+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:02:35.404671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600976+0000","last_change":"2023-10-06T04:32:37.600219+0000","last_active":"2023-10-06T04:32:37.600976+0000","last_peered":"2023-10-06T04:32:37.600976+0000","last_clean":"2023-10-06T04:32:37.600976+0000","last_became_active":"2023-10-06T04:32:37.599968+0000","last_became_peered":"2023-10-06T04:32:37.599968+0000","last_unstale":"2023-10-06T04:32:37.600976+0000","last_undegraded":"2023-10-06T04:32:37.600976+0000","last_fullsized":"2023-10-06T04:32:37.600976+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:38.127232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"34'2","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592552+0000","last_change":"2023-10-06T04:32:24.259250+0000","last_active":"2023-10-06T04:32:37.592552+0000","last_peered":"2023-10-06T04:32:37.592552+0000","last_clean":"2023-10-06T04:32:37.592552+0000","last_became_active":"2023-10-06T04:32:24.258962+0000","last_became_peered":"2023-10-06T04:32:24.258962+0000","last_unstale":"2023-10-06T04:32:37.592552+0000","last_undegraded":"2023-10-06T04:32:37.592552+0000","last_fullsized":"2023-10-06T04:32:37.592552+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593227+0000","last_change":"2023-10-06T04:32:23.445529+0000","last_active":"2023-10-06T04:32:37.593227+0000","last_peered":"2023-10-06T04:32:37.593227+0000","last_clean":"2023-10-06T04:32:37.593227+0000","last_became_active":"2023-10-06T04:32:23.445245+0000","last_became_peered":"2023-10-06T04:32:23.445245+0000","last_unstale":"2023-10-06T04:32:37.593227+0000","last_undegraded":"2023-10-06T04:32:37.593227+0000","last_fullsized":"2023-10-06T04:32:37.593227+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:56:16.408787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601271+0000","last_change":"2023-10-06T04:32:37.600779+0000","last_active":"2023-10-06T04:32:37.601271+0000","last_peered":"2023-10-06T04:32:37.601271+0000","last_clean":"2023-10-06T04:32:37.601271+0000","last_became_active":"2023-10-06T04:32:37.600484+0000","last_became_peered":"2023-10-06T04:32:37.600484+0000","last_unstale":"2023-10-06T04:32:37.601271+0000","last_undegraded":"2023-10-06T04:32:37.601271+0000","last_fullsized":"2023-10-06T04:32:37.601271+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T04:35:31.160924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"34'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596902+0000","last_change":"2023-10-06T04:32:24.270658+0000","last_active":"2023-10-06T04:32:37.596902+0000","last_peered":"2023-10-06T04:32:37.596902+0000","last_clean":"2023-10-06T04:32:37.596902+0000","last_became_active":"2023-10-06T04:32:24.270394+0000","last_became_peered":"2023-10-06T04:32:24.270394+0000","last_unstale":"2023-10-06T04:32:37.596902+0000","last_undegraded":"2023-10-06T04:32:37.596902+0000","last_fullsized":"2023-10-06T04:32:37.596902+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:55:25.294107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596641+0000","last_change":"2023-10-06T04:32:23.452178+0000","last_active":"2023-10-06T04:32:37.596641+0000","last_peered":"2023-10-06T04:32:37.596641+0000","last_clean":"2023-10-06T04:32:37.596641+0000","last_became_active":"2023-10-06T04:32:23.451849+0000","last_became_peered":"2023-10-06T04:32:23.451849+0000","last_unstale":"2023-10-06T04:32:37.596641+0000","last_undegraded":"2023-10-06T04:32:37.596641+0000","last_fullsized":"2023-10-06T04:32:37.596641+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:36:49.026875+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598585+0000","last_change":"2023-10-06T04:32:37.596334+0000","last_active":"2023-10-06T04:32:37.598585+0000","last_peered":"2023-10-06T04:32:37.598585+0000","last_clean":"2023-10-06T04:32:37.598585+0000","last_became_active":"2023-10-06T04:32:37.596085+0000","last_became_peered":"2023-10-06T04:32:37.596085+0000","last_unstale":"2023-10-06T04:32:37.598585+0000","last_undegraded":"2023-10-06T04:32:37.598585+0000","last_fullsized":"2023-10-06T04:32:37.598585+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:01.886035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589791+0000","last_change":"2023-10-06T04:32:24.265320+0000","last_active":"2023-10-06T04:32:37.589791+0000","last_peered":"2023-10-06T04:32:37.589791+0000","last_clean":"2023-10-06T04:32:37.589791+0000","last_became_active":"2023-10-06T04:32:24.264894+0000","last_became_peered":"2023-10-06T04:32:24.264894+0000","last_unstale":"2023-10-06T04:32:37.589791+0000","last_undegraded":"2023-10-06T04:32:37.589791+0000","last_fullsized":"2023-10-06T04:32:37.589791+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:35:56.071063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592944+0000","last_change":"2023-10-06T04:32:23.444136+0000","last_active":"2023-10-06T04:32:37.592944+0000","last_peered":"2023-10-06T04:32:37.592944+0000","last_clean":"2023-10-06T04:32:37.592944+0000","last_became_active":"2023-10-06T04:32:23.443839+0000","last_became_peered":"2023-10-06T04:32:23.443839+0000","last_unstale":"2023-10-06T04:32:37.592944+0000","last_undegraded":"2023-10-06T04:32:37.592944+0000","last_fullsized":"2023-10-06T04:32:37.592944+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:02:10.814479+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.605032+0000","last_change":"2023-10-06T04:32:37.602480+0000","last_active":"2023-10-06T04:32:37.605032+0000","last_peered":"2023-10-06T04:32:37.605032+0000","last_clean":"2023-10-06T04:32:37.605032+0000","last_became_active":"2023-10-06T04:32:37.602207+0000","last_became_peered":"2023-10-06T04:32:37.602207+0000","last_unstale":"2023-10-06T04:32:37.605032+0000","last_undegraded":"2023-10-06T04:32:37.605032+0000","last_fullsized":"2023-10-06T04:32:37.605032+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:15:55.201937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592146+0000","last_change":"2023-10-06T04:32:24.269672+0000","last_active":"2023-10-06T04:32:37.592146+0000","last_peered":"2023-10-06T04:32:37.592146+0000","last_clean":"2023-10-06T04:32:37.592146+0000","last_became_active":"2023-10-06T04:32:24.269407+0000","last_became_peered":"2023-10-06T04:32:24.269407+0000","last_unstale":"2023-10-06T04:32:37.592146+0000","last_undegraded":"2023-10-06T04:32:37.592146+0000","last_fullsized":"2023-10-06T04:32:37.592146+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592605+0000","last_change":"2023-10-06T04:32:23.446685+0000","last_active":"2023-10-06T04:32:37.592605+0000","last_peered":"2023-10-06T04:32:37.592605+0000","last_clean":"2023-10-06T04:32:37.592605+0000","last_became_active":"2023-10-06T04:32:23.446432+0000","last_became_peered":"2023-10-06T04:32:23.446432+0000","last_unstale":"2023-10-06T04:32:37.592605+0000","last_undegraded":"2023-10-06T04:32:37.592605+0000","last_fullsized":"2023-10-06T04:32:37.592605+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:46:33.853623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595478+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:32:37.595478+0000","last_peered":"2023-10-06T04:32:37.595478+0000","last_clean":"2023-10-06T04:32:37.595478+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:32:37.595478+0000","last_undegraded":"2023-10-06T04:32:37.595478+0000","last_fullsized":"2023-10-06T04:32:37.595478+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.603397+0000","last_change":"2023-10-06T04:32:37.599518+0000","last_active":"2023-10-06T04:32:37.603397+0000","last_peered":"2023-10-06T04:32:37.603397+0000","last_clean":"2023-10-06T04:32:37.603397+0000","last_became_active":"2023-10-06T04:32:37.598482+0000","last_became_peered":"2023-10-06T04:32:37.598482+0000","last_unstale":"2023-10-06T04:32:37.603397+0000","last_undegraded":"2023-10-06T04:32:37.603397+0000","last_fullsized":"2023-10-06T04:32:37.603397+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:03:14.898276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"34'7","reported_seq":22,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589246+0000","last_change":"2023-10-06T04:32:24.266084+0000","last_active":"2023-10-06T04:32:37.589246+0000","last_peered":"2023-10-06T04:32:37.589246+0000","last_clean":"2023-10-06T04:32:37.589246+0000","last_became_active":"2023-10-06T04:32:24.265847+0000","last_became_peered":"2023-10-06T04:32:24.265847+0000","last_unstale":"2023-10-06T04:32:37.589246+0000","last_undegraded":"2023-10-06T04:32:37.589246+0000","last_fullsized":"2023-10-06T04:32:37.589246+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:57:22.986130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593234+0000","last_change":"2023-10-06T04:32:23.451106+0000","last_active":"2023-10-06T04:32:37.593234+0000","last_peered":"2023-10-06T04:32:37.593234+0000","last_clean":"2023-10-06T04:32:37.593234+0000","last_became_active":"2023-10-06T04:32:23.450838+0000","last_became_peered":"2023-10-06T04:32:23.450838+0000","last_unstale":"2023-10-06T04:32:37.593234+0000","last_undegraded":"2023-10-06T04:32:37.593234+0000","last_fullsized":"2023-10-06T04:32:37.593234+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597055+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:32:37.597055+0000","last_peered":"2023-10-06T04:32:37.597055+0000","last_clean":"2023-10-06T04:32:37.597055+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:32:37.597055+0000","last_undegraded":"2023-10-06T04:32:37.597055+0000","last_fullsized":"2023-10-06T04:32:37.597055+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.607913+0000","last_change":"2023-10-06T04:32:37.604006+0000","last_active":"2023-10-06T04:32:37.607913+0000","last_peered":"2023-10-06T04:32:37.607913+0000","last_clean":"2023-10-06T04:32:37.607913+0000","last_became_active":"2023-10-06T04:32:37.603823+0000","last_became_peered":"2023-10-06T04:32:37.603823+0000","last_unstale":"2023-10-06T04:32:37.607913+0000","last_undegraded":"2023-10-06T04:32:37.607913+0000","last_fullsized":"2023-10-06T04:32:37.607913+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:10:08.107437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"34'5","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593915+0000","last_change":"2023-10-06T04:32:24.272289+0000","last_active":"2023-10-06T04:32:37.593915+0000","last_peered":"2023-10-06T04:32:37.593915+0000","last_clean":"2023-10-06T04:32:37.593915+0000","last_became_active":"2023-10-06T04:32:24.271996+0000","last_became_peered":"2023-10-06T04:32:24.271996+0000","last_unstale":"2023-10-06T04:32:37.593915+0000","last_undegraded":"2023-10-06T04:32:37.593915+0000","last_fullsized":"2023-10-06T04:32:37.593915+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:01:42.905784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.6","version":"35'3","reported_seq":21,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593048+0000","last_change":"2023-10-06T04:32:23.444286+0000","last_active":"2023-10-06T04:32:37.593048+0000","last_peered":"2023-10-06T04:32:37.593048+0000","last_clean":"2023-10-06T04:32:37.593048+0000","last_became_active":"2023-10-06T04:32:23.443982+0000","last_became_peered":"2023-10-06T04:32:23.443982+0000","last_unstale":"2023-10-06T04:32:37.593048+0000","last_undegraded":"2023-10-06T04:32:37.593048+0000","last_fullsized":"2023-10-06T04:32:37.593048+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:31:20.394340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":31,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.592892+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:32:37.592892+0000","last_peered":"2023-10-06T04:32:37.592892+0000","last_clean":"2023-10-06T04:32:37.592892+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:32:37.592892+0000","last_undegraded":"2023-10-06T04:32:37.592892+0000","last_fullsized":"2023-10-06T04:32:37.592892+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601931+0000","last_change":"2023-10-06T04:32:37.600372+0000","last_active":"2023-10-06T04:32:37.601931+0000","last_peered":"2023-10-06T04:32:37.601931+0000","last_clean":"2023-10-06T04:32:37.601931+0000","last_became_active":"2023-10-06T04:32:37.600140+0000","last_became_peered":"2023-10-06T04:32:37.600140+0000","last_unstale":"2023-10-06T04:32:37.601931+0000","last_undegraded":"2023-10-06T04:32:37.601931+0000","last_fullsized":"2023-10-06T04:32:37.601931+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:20:28.779614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"34'8","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598752+0000","last_change":"2023-10-06T04:32:24.265180+0000","last_active":"2023-10-06T04:32:37.598752+0000","last_peered":"2023-10-06T04:32:37.598752+0000","last_clean":"2023-10-06T04:32:37.598752+0000","last_became_active":"2023-10-06T04:32:24.264769+0000","last_became_peered":"2023-10-06T04:32:24.264769+0000","last_unstale":"2023-10-06T04:32:37.598752+0000","last_undegraded":"2023-10-06T04:32:37.598752+0000","last_fullsized":"2023-10-06T04:32:37.598752+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:38:24.966729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":4194304,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":4096,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":17,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593995+0000","last_change":"2023-10-06T04:32:23.452281+0000","last_active":"2023-10-06T04:32:37.593995+0000","last_peered":"2023-10-06T04:32:37.593995+0000","last_clean":"2023-10-06T04:32:37.593995+0000","last_became_active":"2023-10-06T04:32:23.452023+0000","last_became_peered":"2023-10-06T04:32:23.452023+0000","last_unstale":"2023-10-06T04:32:37.593995+0000","last_undegraded":"2023-10-06T04:32:37.593995+0000","last_fullsized":"2023-10-06T04:32:37.593995+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T05:29:35.941426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":28,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597399+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:32:37.597399+0000","last_peered":"2023-10-06T04:32:37.597399+0000","last_clean":"2023-10-06T04:32:37.597399+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:32:37.597399+0000","last_undegraded":"2023-10-06T04:32:37.597399+0000","last_fullsized":"2023-10-06T04:32:37.597399+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":182,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594956+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:32:37.594956+0000","last_peered":"2023-10-06T04:32:37.594956+0000","last_clean":"2023-10-06T04:32:37.594956+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:32:37.594956+0000","last_undegraded":"2023-10-06T04:32:37.594956+0000","last_fullsized":"2023-10-06T04:32:37.594956+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600163+0000","last_change":"2023-10-06T04:32:37.599423+0000","last_active":"2023-10-06T04:32:37.600163+0000","last_peered":"2023-10-06T04:32:37.600163+0000","last_clean":"2023-10-06T04:32:37.600163+0000","last_became_active":"2023-10-06T04:32:37.599103+0000","last_became_peered":"2023-10-06T04:32:37.599103+0000","last_unstale":"2023-10-06T04:32:37.600163+0000","last_undegraded":"2023-10-06T04:32:37.600163+0000","last_fullsized":"2023-10-06T04:32:37.600163+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:15:01.505070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"34'9","reported_seq":24,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594972+0000","last_change":"2023-10-06T04:32:24.267047+0000","last_active":"2023-10-06T04:32:37.594972+0000","last_peered":"2023-10-06T04:32:37.594972+0000","last_clean":"2023-10-06T04:32:37.594972+0000","last_became_active":"2023-10-06T04:32:24.266777+0000","last_became_peered":"2023-10-06T04:32:24.266777+0000","last_unstale":"2023-10-06T04:32:37.594972+0000","last_undegraded":"2023-10-06T04:32:37.594972+0000","last_fullsized":"2023-10-06T04:32:37.594972+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:23.977882+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"35'3","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.598932+0000","last_change":"2023-10-06T04:32:23.448480+0000","last_active":"2023-10-06T04:32:37.598932+0000","last_peered":"2023-10-06T04:32:37.598932+0000","last_clean":"2023-10-06T04:32:37.598932+0000","last_became_active":"2023-10-06T04:32:23.448242+0000","last_became_peered":"2023-10-06T04:32:23.448242+0000","last_unstale":"2023-10-06T04:32:37.598932+0000","last_undegraded":"2023-10-06T04:32:37.598932+0000","last_fullsized":"2023-10-06T04:32:37.598932+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:04:15.866399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594875+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:32:37.594875+0000","last_peered":"2023-10-06T04:32:37.594875+0000","last_clean":"2023-10-06T04:32:37.594875+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:32:37.594875+0000","last_undegraded":"2023-10-06T04:32:37.594875+0000","last_fullsized":"2023-10-06T04:32:37.594875+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.604733+0000","last_change":"2023-10-06T04:32:37.602301+0000","last_active":"2023-10-06T04:32:37.604733+0000","last_peered":"2023-10-06T04:32:37.604733+0000","last_clean":"2023-10-06T04:32:37.604733+0000","last_became_active":"2023-10-06T04:32:37.601941+0000","last_became_peered":"2023-10-06T04:32:37.601941+0000","last_unstale":"2023-10-06T04:32:37.604733+0000","last_undegraded":"2023-10-06T04:32:37.604733+0000","last_fullsized":"2023-10-06T04:32:37.604733+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T10:47:01.886035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"34'5","reported_seq":20,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.596220+0000","last_change":"2023-10-06T04:32:24.266050+0000","last_active":"2023-10-06T04:32:37.596220+0000","last_peered":"2023-10-06T04:32:37.596220+0000","last_clean":"2023-10-06T04:32:37.596220+0000","last_became_active":"2023-10-06T04:32:24.265769+0000","last_became_peered":"2023-10-06T04:32:24.265769+0000","last_unstale":"2023-10-06T04:32:37.596220+0000","last_undegraded":"2023-10-06T04:32:37.596220+0000","last_fullsized":"2023-10-06T04:32:37.596220+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T11:55:01.427293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":18,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594889+0000","last_change":"2023-10-06T04:32:23.443114+0000","last_active":"2023-10-06T04:32:37.594889+0000","last_peered":"2023-10-06T04:32:37.594889+0000","last_clean":"2023-10-06T04:32:37.594889+0000","last_became_active":"2023-10-06T04:32:23.442844+0000","last_became_peered":"2023-10-06T04:32:23.442844+0000","last_unstale":"2023-10-06T04:32:37.594889+0000","last_undegraded":"2023-10-06T04:32:37.594889+0000","last_fullsized":"2023-10-06T04:32:37.594889+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T16:12:26.153742+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.594004+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:32:37.594004+0000","last_peered":"2023-10-06T04:32:37.594004+0000","last_clean":"2023-10-06T04:32:37.594004+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:32:37.594004+0000","last_undegraded":"2023-10-06T04:32:37.594004+0000","last_fullsized":"2023-10-06T04:32:37.594004+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.597853+0000","last_change":"2023-10-06T04:32:37.595103+0000","last_active":"2023-10-06T04:32:37.597853+0000","last_peered":"2023-10-06T04:32:37.597853+0000","last_clean":"2023-10-06T04:32:37.597853+0000","last_became_active":"2023-10-06T04:32:37.594835+0000","last_became_peered":"2023-10-06T04:32:37.594835+0000","last_unstale":"2023-10-06T04:32:37.597853+0000","last_undegraded":"2023-10-06T04:32:37.597853+0000","last_fullsized":"2023-10-06T04:32:37.597853+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:37:44.627334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"34'8","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.601888+0000","last_change":"2023-10-06T04:32:24.265505+0000","last_active":"2023-10-06T04:32:37.601888+0000","last_peered":"2023-10-06T04:32:37.601888+0000","last_clean":"2023-10-06T04:32:37.601888+0000","last_became_active":"2023-10-06T04:32:24.265228+0000","last_became_peered":"2023-10-06T04:32:24.265228+0000","last_unstale":"2023-10-06T04:32:37.601888+0000","last_undegraded":"2023-10-06T04:32:37.601888+0000","last_fullsized":"2023-10-06T04:32:37.601888+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:35:00.380436+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.600560+0000","last_change":"2023-10-06T04:32:23.438229+0000","last_active":"2023-10-06T04:32:37.600560+0000","last_peered":"2023-10-06T04:32:37.600560+0000","last_clean":"2023-10-06T04:32:37.600560+0000","last_became_active":"2023-10-06T04:32:23.437954+0000","last_became_peered":"2023-10-06T04:32:23.437954+0000","last_unstale":"2023-10-06T04:32:37.600560+0000","last_undegraded":"2023-10-06T04:32:37.600560+0000","last_fullsized":"2023-10-06T04:32:37.600560+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:12:22.972008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":29,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595851+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:32:37.595851+0000","last_peered":"2023-10-06T04:32:37.595851+0000","last_clean":"2023-10-06T04:32:37.595851+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:32:37.595851+0000","last_undegraded":"2023-10-06T04:32:37.595851+0000","last_fullsized":"2023-10-06T04:32:37.595851+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":9,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.607574+0000","last_change":"2023-10-06T04:32:37.601364+0000","last_active":"2023-10-06T04:32:37.607574+0000","last_peered":"2023-10-06T04:32:37.607574+0000","last_clean":"2023-10-06T04:32:37.607574+0000","last_became_active":"2023-10-06T04:32:37.600999+0000","last_became_peered":"2023-10-06T04:32:37.600999+0000","last_unstale":"2023-10-06T04:32:37.607574+0000","last_undegraded":"2023-10-06T04:32:37.607574+0000","last_fullsized":"2023-10-06T04:32:37.607574+0000","mapping_epoch":40,"log_start":"0'0","ondisk_log_start":"0'0","created":40,"last_epoch_clean":41,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:36.583838+0000","last_clean_scrub_stamp":"2023-10-06T04:32:36.583838+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T15:13:42.253389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"34'7","reported_seq":23,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.593094+0000","last_change":"2023-10-06T04:32:24.272223+0000","last_active":"2023-10-06T04:32:37.593094+0000","last_peered":"2023-10-06T04:32:37.593094+0000","last_clean":"2023-10-06T04:32:37.593094+0000","last_became_active":"2023-10-06T04:32:24.271905+0000","last_became_peered":"2023-10-06T04:32:24.271905+0000","last_unstale":"2023-10-06T04:32:37.593094+0000","last_undegraded":"2023-10-06T04:32:37.593094+0000","last_fullsized":"2023-10-06T04:32:37.593094+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:23.423335+0000","last_clean_scrub_stamp":"2023-10-06T04:32:23.423335+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:33:17.120222+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":16,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.589921+0000","last_change":"2023-10-06T04:32:23.444321+0000","last_active":"2023-10-06T04:32:37.589921+0000","last_peered":"2023-10-06T04:32:37.589921+0000","last_clean":"2023-10-06T04:32:37.589921+0000","last_became_active":"2023-10-06T04:32:23.444073+0000","last_became_peered":"2023-10-06T04:32:23.444073+0000","last_unstale":"2023-10-06T04:32:37.589921+0000","last_undegraded":"2023-10-06T04:32:37.589921+0000","last_fullsized":"2023-10-06T04:32:37.589921+0000","mapping_epoch":31,"log_start":"0'0","ondisk_log_start":"0'0","created":31,"last_epoch_clean":32,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:32:22.417461+0000","last_clean_scrub_stamp":"2023-10-06T04:32:22.417461+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:41:37.675115+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":30,"reported_epoch":41,"state":"active+clean","last_fresh":"2023-10-06T04:32:37.595148+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:32:37.595148+0000","last_peered":"2023-10-06T04:32:37.595148+0000","last_clean":"2023-10-06T04:32:37.595148+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:32:37.595148+0000","last_undegraded":"2023-10-06T04:32:37.595148+0000","last_fullsized":"2023-10-06T04:32:37.595148+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":8},{"poolid":6,"num_pg":64,"stat_sum":{"num_bytes":6291456,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":12582912,"data_stored":12582912,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":504,"ondisk_log_size":504,"up":128,"acting":128,"num_store_stats":8},{"poolid":5,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":1,"num_read_kb":0,"num_write":92,"num_write_kb":36,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":81,"ondisk_log_size":81,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574868,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300388,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071452,"statfs":{"total":96636764160,"available":96329166848,"internally_reserved":0,"allocated":3174400,"data_stored":2897698,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.438},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.46400000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.58999999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.67300000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.65900000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.63},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.73299999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.751}]}]},{"osd":6,"up_from":13,"seq":55834574867,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.44500000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.48999999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.54300000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.51900000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.41299999999999998},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.45800000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.378},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.501}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.55800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574868,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.28899999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.34699999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.23200000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.39400000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.311}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.42399999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.502}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.52400000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.63800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298340,"kb_used_data":1052,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073500,"statfs":{"total":96636764160,"available":96331264000,"internally_reserved":0,"allocated":1077248,"data_stored":800546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.27600000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.34399999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.38700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.36299999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.40600000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.64700000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.57599999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.66000000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574868,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.246}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.26600000000000001},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.28499999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.35799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.47799999999999998},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.374}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.45100000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.42899999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.38300000000000001},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.441}]}]},{"osd":3,"up_from":13,"seq":55834574868,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301856,"kb_used_data":4568,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069984,"statfs":{"total":96636764160,"available":96327663616,"internally_reserved":0,"allocated":4677632,"data_stored":4404482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.39000000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.375}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.44900000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.63700000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.60499999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.70899999999999996}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.68300000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574869,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297760,"kb_used_data":472,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074080,"statfs":{"total":96636764160,"available":96331857920,"internally_reserved":0,"allocated":483328,"data_stored":210178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.56499999999999995}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.44700000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.496}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.46000000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.373}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.59699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.61899999999999999}]}]},{"osd":5,"up_from":13,"seq":55834574868,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303904,"kb_used_data":6616,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067936,"statfs":{"total":96636764160,"available":96325566464,"internally_reserved":0,"allocated":6774784,"data_stored":6501634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.68899999999999995}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.53600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.76700000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.67100000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.74399999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.82599999999999996},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.71899999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4194304,"data_stored":4194304,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":6291456,"data_stored":6291456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":2097152,"data_stored":2097152,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:32:46.544 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-10-06T04:32:46.544 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph -s 2023-10-06T04:32:46.690 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.688+0000 7f1b2c43d700 1 Processor -- start 2023-10-06T04:32:46.691 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.688+0000 7f1b2c43d700 1 -- start start 2023-10-06T04:32:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b2c43d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1b2406a150 0x7f1b24162160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b2c43d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b240afdf0 con 0x7f1b240aeb50 2023-10-06T04:32:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b2c43d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b2415af60 con 0x7f1b24158440 2023-10-06T04:32:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b2c43d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b2415b900 con 0x7f1b2406a150 2023-10-06T04:32:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b2a1d9700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1b240aeb50 0x7f1b240af5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:45060/0 (socket says 172.21.15.123:45060) 2023-10-06T04:32:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b2a1d9700 1 -- 172.21.15.123:0/2447172407 learned_addr learned my addr 172.21.15.123:0/2447172407 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:46.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b2a9da700 1 --2- 172.21.15.123:0/2447172407 >> v2:172.21.15.177:3300/0 conn(0x7f1b2406a150 0x7f1b24162160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.689+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2225281167 0 0) 0x7f1b240afdf0 con 0x7f1b240aeb50 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b10003570 con 0x7f1b240aeb50 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100318274 0 0) 0x7f1b10003570 con 0x7f1b240aeb50 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 >> v2:172.21.15.177:3300/0 conn(0x7f1b2406a150 msgr2=0x7f1b24162160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 --2- 172.21.15.123:0/2447172407 >> v2:172.21.15.177:3300/0 conn(0x7f1b2406a150 0x7f1b24162160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 >> v1:172.21.15.177:6789/0 conn(0x7f1b24158440 legacy=0x7f1b24069a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2415b240 con 0x7f1b240aeb50 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f1b14001470 con 0x7f1b240aeb50 2023-10-06T04:32:46.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 >> v1:172.21.15.123:6789/0 conn(0x7f1b240aeb50 legacy=0x7f1b240af5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10003a60 0x7f1b10003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b10004810 0x7f1b10004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b100051c0 0x7f1b1000ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b2415b900 con 0x7f1b10004810 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1000c0d0 con 0x7f1b100051c0 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b1000c290 con 0x7f1b10003a60 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b299d8700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b10004810 0x7f1b10004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b2a1d9700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10003a60 0x7f1b10003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b2a9da700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b100051c0 0x7f1b1000ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b299d8700 1 -- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10003a60 msgr2=0x7f1b10003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b299d8700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10003a60 0x7f1b10003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b299d8700 1 -- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b100051c0 msgr2=0x7f1b1000ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b299d8700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b100051c0 0x7f1b1000ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b299d8700 1 -- 172.21.15.123:0/2447172407 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b10003570 con 0x7f1b10004810 2023-10-06T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b2a9da700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b100051c0 0x7f1b1000ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b2a9da700 1 -- 172.21.15.123:0/2447172407 reap_dead start 2023-10-06T04:32:46.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b299d8700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b10004810 0x7f1b10004c80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1b200018e0 tx=0x7f1b20007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=459be133634d7fbd server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20012420 con 0x7f1b10004810 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b200125a0 con 0x7f1b10004810 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.690+0000 7f1b291d7700 1 -- 172.21.15.123:0/2447172407 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20012960 con 0x7f1b10004810 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.691+0000 7f1b2c43d700 1 -- 172.21.15.123:0/2447172407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b10004810 msgr2=0x7f1b10004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.691+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b10004810 0x7f1b10004c80 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7f1b200018e0 tx=0x7f1b20007d30 comp rx=0 tx=0).stop 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.691+0000 7f1b2c43d700 1 -- 172.21.15.123:0/2447172407 shutdown_connections 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.691+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/2447172407 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b10004810 0x7f1b10004c80 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.691+0000 7f1b2c43d700 1 -- 172.21.15.123:0/2447172407 >> 172.21.15.123:0/2447172407 conn(0x7f1b240c9180 msgr2=0x7f1b240c9580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:46.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.691+0000 7f1b2c43d700 1 -- 172.21.15.123:0/2447172407 shutdown_connections 2023-10-06T04:32:46.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.691+0000 7f1b2c43d700 1 -- 172.21.15.123:0/2447172407 wait complete. 2023-10-06T04:32:46.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.692+0000 7f1b2c43d700 1 Processor -- start 2023-10-06T04:32:46.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.692+0000 7f1b2c43d700 1 -- start start 2023-10-06T04:32:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2c43d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10004810 0x7f1b24153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2c43d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b240c1910 0x7f1b240c7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2c43d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b24169880 0x7f1b24169d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2c43d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240c8d70 con 0x7f1b240c1910 2023-10-06T04:32:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2c43d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b240c88e0 con 0x7f1b24169880 2023-10-06T04:32:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2c43d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b24153ad0 con 0x7f1b10004810 2023-10-06T04:32:46.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b299d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b240c1910 0x7f1b240c7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b299d8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b240c1910 0x7f1b240c7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33686/0 (socket says 172.21.15.123:33686) 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b299d8700 1 -- 172.21.15.123:0/322931926 learned_addr learned my addr 172.21.15.123:0/322931926 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2a1d9700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10004810 0x7f1b24153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.693+0000 7f1b2a9da700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b24169880 0x7f1b24169d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b299d8700 1 -- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10004810 msgr2=0x7f1b24153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b299d8700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10004810 0x7f1b24153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b299d8700 1 -- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b24169880 msgr2=0x7f1b24169d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b299d8700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b24169880 0x7f1b24169d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b299d8700 1 -- 172.21.15.123:0/322931926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b2416a260 con 0x7f1b240c1910 2023-10-06T04:32:46.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b299d8700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b240c1910 0x7f1b240c7480 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1b20012ad0 tx=0x7f1b20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b1a7fc700 1 -- 172.21.15.123:0/322931926 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b2000ca40 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b1a7fc700 1 -- 172.21.15.123:0/322931926 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2000cbc0 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b240c7cd0 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b1a7fc700 1 -- 172.21.15.123:0/322931926 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b20006820 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.694+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b240c8290 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.695+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b240c8a50 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.695+0000 7f1b1a7fc700 1 -- 172.21.15.123:0/322931926 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1b20019030 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.696+0000 7f1b1a7fc700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.696+0000 7f1b1a7fc700 1 -- 172.21.15.123:0/322931926 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f1b20085e20 con 0x7f1b240c1910 2023-10-06T04:32:46.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.696+0000 7f1b2a1d9700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:46.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.696+0000 7f1b2a1d9700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b08066f30 0x7f1b080693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b140015a0 tx=0x7f1b14001e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:46.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.700+0000 7f1b1a7fc700 1 -- 172.21.15.123:0/322931926 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1b20052dd0 con 0x7f1b240c1910 2023-10-06T04:32:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.955+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "status"} v 0) v1 -- 0x7f1b2404bb80 con 0x7f1b240c1910 2023-10-06T04:32:46.958 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.956+0000 7f1b1a7fc700 1 -- 172.21.15.123:0/322931926 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) v1 ==== 54+0+548 (secure 0 0 0) 0x7f1b20017070 con 0x7f1b240c1910 2023-10-06T04:32:46.958 INFO:teuthology.orchestra.run.smithi123.stdout: cluster: 2023-10-06T04:32:46.958 INFO:teuthology.orchestra.run.smithi123.stdout: id: 1cea82d3-d2ef-4e20-8c3e-67296f7df92c 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: health: HEALTH_ERR 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: 2 filesystems are degraded 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: 2 filesystems are offline 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: services: 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: mon: 3 daemons, quorum a,b,c (age 97s) 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: mgr: y(active, since 92s), standbys: x 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: mds: 0/2 daemons up (2 failed), 4 standby 2023-10-06T04:32:46.959 INFO:teuthology.orchestra.run.smithi123.stdout: osd: 8 osds: 8 up (since 86s), 8 in (since 94s) 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: data: 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: volumes: 0/2 healthy, 2 failed 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: pools: 5 pools, 137 pgs 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: objects: 4 objects, 6.6 MiB 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: usage: 2.3 GiB used, 718 GiB / 720 GiB avail 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: pgs: 137 active+clean 2023-10-06T04:32:46.960 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:46.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b08066f30 msgr2=0x7f1b080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b08066f30 0x7f1b080693e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f1b140015a0 tx=0x7f1b14001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b240c1910 msgr2=0x7f1b240c7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b240c1910 0x7f1b240c7480 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f1b20012ad0 tx=0x7f1b20015000 comp rx=0 tx=0).stop 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 shutdown_connections 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1b08066f30 0x7f1b080693e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1b24169880 0x7f1b24169d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1b10004810 0x7f1b24153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 --2- 172.21.15.123:0/322931926 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1b240c1910 0x7f1b240c7480 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 >> 172.21.15.123:0/322931926 conn(0x7f1b240c9180 msgr2=0x7f1b2406ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 shutdown_connections 2023-10-06T04:32:46.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:46.958+0000 7f1b2c43d700 1 -- 172.21.15.123:0/322931926 wait complete. 2023-10-06T04:32:46.972 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 init --force-init --filesystem recovery_fs --alternate-pool recovery_fs_metadata 2023-10-06T04:32:47.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.012+0000 7f9fdab0e740 1 -- start start 2023-10-06T04:32:47.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.012+0000 7f9fdab0e740 1 Processor -- start 2023-10-06T04:32:47.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.012+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0a400 0x55b46b642580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46aa0a800 0x55b46b642b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0ac00 0x55b46b643080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d5cc0 con 0x55b46aa0a400 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d5e00 con 0x55b46aa0ac00 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d5540 con 0x55b46aa0a800 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0a400 0x55b46b642580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0a400 0x55b46b642580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33702/0 (socket says 172.21.15.123:33702) 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/3400471895 learned_addr learned my addr 172.21.15.123:0/3400471895 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd355e700 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46aa0a800 0x55b46b642b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd4560700 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0ac00 0x55b46b643080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46aa0a800 msgr2=0x55b46b642b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46aa0a800 0x55b46b642b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0ac00 msgr2=0x55b46b643080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0ac00 0x55b46b643080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/3400471895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46a8b8340 con 0x55b46aa0a400 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0a400 0x55b46b642580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55b46b666630 tx=0x55b46b627a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b0b36db8e531be server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd155a700 1 -- 172.21.15.123:0/3400471895 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b46a8cd080 con 0x55b46aa0a400 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd155a700 1 -- 172.21.15.123:0/3400471895 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b46a8eaf00 con 0x55b46aa0a400 2023-10-06T04:32:47.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd155a700 1 -- 172.21.15.123:0/3400471895 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b46a8cd600 con 0x55b46aa0a400 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 1 -- 172.21.15.123:0/3400471895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b46a8b8000 con 0x55b46aa0a400 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 1 -- 172.21.15.123:0/3400471895 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55b46a8b89c0 con 0x55b46aa0a400 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fdab0e740 4 waiting for MDS map... 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.013+0000 7f9fd155a700 1 -- 172.21.15.123:0/3400471895 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55b46a8d8c40 con 0x55b46aa0a400 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.014+0000 7f9fd155a700 1 -- 172.21.15.123:0/3400471895 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x55b46b69d200 con 0x55b46aa0a400 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.014+0000 7f9fdab0e740 4 Got MDS map 22 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.014+0000 7f9fdab0e740 4 datascan.main: Using metadata pool output 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.014+0000 7f9fdab0e740 4 datascan.main: connecting to RADOS... 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.014+0000 7f9fdab0e740 1 Processor -- start 2023-10-06T04:32:47.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.014+0000 7f9fdab0e740 1 -- start start 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0b800 0x55b46b643600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0bc00 0x55b46b643b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b6e4000 0x55b46b644100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d5b80 con 0x55b46aa0bc00 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d5e00 con 0x55b46aa0b800 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d5cc0 con 0x55b46b6e4000 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0bc00 0x55b46b643b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0bc00 0x55b46b643b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33710/0 (socket says 172.21.15.123:33710) 2023-10-06T04:32:47.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/1079218693 learned_addr learned my addr 172.21.15.123:0/1079218693 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd355e700 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b6e4000 0x55b46b644100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd4560700 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0b800 0x55b46b643600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b6e4000 msgr2=0x55b46b644100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b6e4000 0x55b46b644100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0b800 msgr2=0x55b46b643600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0b800 0x55b46b643600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/1079218693 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46a8b8340 con 0x55b46aa0bc00 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd4560700 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0b800 0x55b46b643600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd355e700 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b6e4000 0x55b46b644100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.027 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.015+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0bc00 0x55b46b643b80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55b46b667560 tx=0x55b46b627800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498b0605e64621c5 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fcf556700 1 -- 172.21.15.123:0/1079218693 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b46a8cd1e0 con 0x55b46aa0bc00 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fcf556700 1 -- 172.21.15.123:0/1079218693 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b46b6a6780 con 0x55b46aa0bc00 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fcf556700 1 -- 172.21.15.123:0/1079218693 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b46a8ccf20 con 0x55b46aa0bc00 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 -- 172.21.15.123:0/1079218693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0bc00 msgr2=0x55b46b643b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0bc00 0x55b46b643b80 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x55b46b667560 tx=0x55b46b627800 comp rx=0 tx=0).stop 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 -- 172.21.15.123:0/1079218693 shutdown_connections 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b6e4000 0x55b46b644100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0b800 0x55b46b643600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.028 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/1079218693 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0bc00 0x55b46b643b80 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 -- 172.21.15.123:0/1079218693 >> 172.21.15.123:0/1079218693 conn(0x55b46aa0b000 msgr2=0x55b46a969800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 -- 172.21.15.123:0/1079218693 shutdown_connections 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.016+0000 7f9fdab0e740 1 -- 172.21.15.123:0/1079218693 wait complete. 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 Processor -- start 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 -- start start 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0bc00 0x55b46b644c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0b800 0x55b46b644680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b704000 0x55b46b644100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d4dc0 con 0x55b46aa0b800 2023-10-06T04:32:47.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d43c0 con 0x55b46aa0bc00 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fdab0e740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b46a8d4000 con 0x55b46b704000 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fd3d5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0b800 0x55b46b644680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fd3d5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0b800 0x55b46b644680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33714/0 (socket says 172.21.15.123:33714) 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/2096699278 learned_addr learned my addr 172.21.15.123:0/2096699278 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fd355e700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b704000 0x55b46b644100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.017+0000 7f9fd4560700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0bc00 0x55b46b644c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b704000 msgr2=0x55b46b644100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46b704000 0x55b46b644100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0bc00 msgr2=0x55b46b644c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0bc00 0x55b46b644c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b46a8b89c0 con 0x55b46aa0b800 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fd4560700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0bc00 0x55b46b644c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0b800 0x55b46b644680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55b46b6677d0 tx=0x55b46b6fa300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fcdd53700 1 -- 172.21.15.123:0/2096699278 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b46a8cd1e0 con 0x55b46aa0b800 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fcdd53700 1 -- 172.21.15.123:0/2096699278 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b46a8ea000 con 0x55b46aa0b800 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55b46a8b8000 con 0x55b46aa0b800 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fcdd53700 1 -- 172.21.15.123:0/2096699278 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55b46a8ccf20 con 0x55b46aa0b800 2023-10-06T04:32:47.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55b46a8b8ea0 con 0x55b46aa0b800 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.018+0000 7f9fdab0e740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.019+0000 7f9fcdd53700 1 -- 172.21.15.123:0/2096699278 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55b46b60b080 con 0x55b46aa0b800 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.019+0000 7f9fcdd53700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55b46b70a800 0x55b46b645700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.019+0000 7f9fcdd53700 1 -- 172.21.15.123:0/2096699278 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55b46a8d8a80 con 0x55b46aa0b800 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fd4560700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55b46b70a800 0x55b46b645700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55b46b704400 0x55b46b643b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55b46b704800 con 0x55b46b704400 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fd355e700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55b46b704400 0x55b46b643b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fd4560700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55b46b70a800 0x55b46b645700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55b46b60fb90 tx=0x55b46b6fa2a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fd355e700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55b46b704400 0x55b46b643b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fd355e700 1 -- 172.21.15.123:0/2096699278 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== osd_op_reply(1 1.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 160+0+0 (crc 0 0 0) 0x55b46b664fc0 con 0x55b46b704400 2023-10-06T04:32:47.033 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.020+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:2 7.14 7:287034c4:::1.00000000.inode:head [writefull 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55b46b704c00 con 0x55b46b704400 2023-10-06T04:32:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.022+0000 7f9fd355e700 1 -- 172.21.15.123:0/2096699278 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== osd_op_reply(2 1.00000000.inode [writefull 0~554] v42'1 uv1 ondisk = 0) v8 ==== 160+0+0 (crc 0 0 0) 0x55b46b664fc0 con 0x55b46b704400 2023-10-06T04:32:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.022+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55b46b705000 0x55b46b643600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.022+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:3 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55b46b705400 con 0x55b46b705000 2023-10-06T04:32:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.022+0000 7f9fd4560700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55b46b705000 0x55b46b643600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.022+0000 7f9fd4560700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55b46b705000 0x55b46b643600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.023+0000 7f9fd4560700 1 -- 172.21.15.123:0/2096699278 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== osd_op_reply(3 100.00000000.inode [stat] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 162+0+0 (crc 0 0 0) 0x55b46b665680 con 0x55b46b705000 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.023+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:4 7.1f 7:fe07bba1:::100.00000000.inode:head [writefull 0~554 in=554b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55b46b705800 con 0x55b46b705000 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.024+0000 7f9fd4560700 1 -- 172.21.15.123:0/2096699278 <== osd.6 v2:172.21.15.177:6816/119976 2 ==== osd_op_reply(4 100.00000000.inode [writefull 0~554] v42'1 uv1 ondisk = 0) v8 ==== 162+0+0 (crc 0 0 0) 0x55b46b665680 con 0x55b46b705000 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.024+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55b46b705c00 0x55b46b80c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.024+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:5 7.13 7:cd5a64a3:::100.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55b46b70ac00 con 0x55b46b705c00 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.025+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55b46b705c00 0x55b46b80c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.025+0000 7f9fd3d5f700 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55b46b705c00 0x55b46b80c000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.025+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/2096699278 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(5 100.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55b46b665d40 con 0x55b46b705c00 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.025+0000 7f9fdab0e740 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.025+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:6 7.13 7:cd5a64a3:::100.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55b46b70b000 con 0x55b46b705c00 2023-10-06T04:32:47.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fd3d5f700 1 -- 172.21.15.123:0/2096699278 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== osd_op_reply(6 100.00000000 [create,omap-set-header] v42'1 uv1 ondisk = 0) v8 ==== 198+0+0 (crc 0 0 0) 0x55b46b665d40 con 0x55b46b705c00 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x100 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55b46b705c00 msgr2=0x55b46b80c000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55b46b705c00 0x55b46b80c000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55b46b704400 msgr2=0x55b46b643b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55b46b704400 0x55b46b643b80 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55b46b705000 msgr2=0x55b46b643600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55b46b705000 0x55b46b643600 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fd355e700 1 -- 172.21.15.123:0/2096699278 reap_dead start 2023-10-06T04:32:47.039 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55b46b70a800 msgr2=0x55b46b645700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55b46b70a800 0x55b46b645700 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x55b46b60fb90 tx=0x55b46b6fa2a0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0b800 msgr2=0x55b46b644680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.026+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0b800 0x55b46b644680 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x55b46b6677d0 tx=0x55b46b6fa300 comp rx=0 tx=0).stop 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 shutdown_connections 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0b800 0x55b46b644680 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/2096699278 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55b46b70a800 0x55b46b645700 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 >> 172.21.15.123:0/2096699278 conn(0x55b46aa0b000 msgr2=0x55b46a969800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 shutdown_connections 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/2096699278 wait complete. 2023-10-06T04:32:47.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/3400471895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0a400 msgr2=0x55b46b642580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0a400 0x55b46b642580 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x55b46b666630 tx=0x55b46b627a40 comp rx=0 tx=0).stop 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/3400471895 shutdown_connections 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55b46aa0a400 0x55b46b642580 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55b46aa0ac00 0x55b46b643080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 --2- 172.21.15.123:0/3400471895 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55b46aa0a800 0x55b46b642b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/3400471895 >> 172.21.15.123:0/3400471895 conn(0x55b46aa0a000 msgr2=0x55b46a969000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/3400471895 shutdown_connections 2023-10-06T04:32:47.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.027+0000 7f9fdab0e740 1 -- 172.21.15.123:0/3400471895 wait complete. 2023-10-06T04:32:47.041 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'init', '--force-init', '--filesystem', 'recovery_fs', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.069482, result: 2023-10-06T04:32:47.042 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset session 2023-10-06T04:32:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 -- start start 2023-10-06T04:32:47.115 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 Processor -- start 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b6800 0x5596171f8b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596165b6c00 0x5596171f9080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7000 0x5596171f9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559616481cc0 con 0x5596165b6c00 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559616481540 con 0x5596165b6800 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b9a558280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559616481e00 con 0x5596165b7000 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b93208700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596165b6c00 0x5596171f9080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b93208700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596165b6c00 0x5596171f9080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33738/0 (socket says 172.21.15.123:33738) 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b93208700 1 -- 172.21.15.123:0/2739569417 learned_addr learned my addr 172.21.15.123:0/2739569417 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.122 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.113+0000 7f5b93a09700 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b6800 0x5596171f8b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b9420a700 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7000 0x5596171f9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b93208700 1 -- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7000 msgr2=0x5596171f9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b93208700 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7000 0x5596171f9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b93208700 1 -- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b6800 msgr2=0x5596171f8b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b93208700 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b6800 0x5596171f8b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b93208700 1 -- 172.21.15.123:0/2739569417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559616464340 con 0x5596165b6c00 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b93208700 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596165b6c00 0x5596171f9080 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5596171cdc50 tx=0x5596171d33e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b037aa9bec67e70a server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b91204700 1 -- 172.21.15.123:0/2739569417 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559616479600 con 0x5596165b6c00 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b91204700 1 -- 172.21.15.123:0/2739569417 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559616496600 con 0x5596165b6c00 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b91204700 1 -- 172.21.15.123:0/2739569417 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5596164791e0 con 0x5596165b6c00 2023-10-06T04:32:47.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b9a558280 1 -- 172.21.15.123:0/2739569417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559616464000 con 0x5596165b6c00 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b9a558280 1 -- 172.21.15.123:0/2739569417 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x559616464ea0 con 0x5596165b6c00 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b9a558280 4 waiting for MDS map... 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b91204700 1 -- 172.21.15.123:0/2739569417 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x559616484c40 con 0x5596165b6c00 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b91204700 1 -- 172.21.15.123:0/2739569417 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x55961720a600 con 0x5596165b6c00 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b9a558280 4 Got MDS map 22 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.114+0000 7f5b9a558280 10 main: main 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.115+0000 7f5b9a558280 4 main: connecting to RADOS... 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.115+0000 7f5b9a558280 1 Processor -- start 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.115+0000 7f5b9a558280 1 -- start start 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b7c00 0x5596171f9b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559617292000 0x5596171fa100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.124 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fa680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9a558280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x559616481b80 con 0x559617292400 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9a558280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559616481540 con 0x5596165b7c00 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9a558280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559616480dc0 con 0x559617292000 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fa680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33752/0 (socket says 172.21.15.123:33752) 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 -- 172.21.15.123:0/2116394635 learned_addr learned my addr 172.21.15.123:0/2116394635 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93a09700 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559617292000 0x5596171fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9420a700 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b7c00 0x5596171f9b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 -- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559617292000 msgr2=0x5596171fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559617292000 0x5596171fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 -- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b7c00 msgr2=0x5596171f9b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.125 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b7c00 0x5596171f9b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 -- 172.21.15.123:0/2116394635 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559616465860 con 0x559617292400 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b9420a700 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b7c00 0x5596171f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b93208700 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fa680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5596172291a0 tx=0x5596171c28a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71b31c40053ca75f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b8f200700 1 -- 172.21.15.123:0/2116394635 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559616478f20 con 0x559617292400 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b8f200700 1 -- 172.21.15.123:0/2116394635 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559616496000 con 0x559617292400 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.116+0000 7f5b8f200700 1 -- 172.21.15.123:0/2116394635 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559616479600 con 0x559617292400 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 -- 172.21.15.123:0/2116394635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 msgr2=0x5596171fa680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fa680 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x5596172291a0 tx=0x5596171c28a0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 -- 172.21.15.123:0/2116394635 shutdown_connections 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fa680 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b7c00 0x5596171f9b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.126 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2116394635 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x559617292000 0x5596171fa100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 -- 172.21.15.123:0/2116394635 >> 172.21.15.123:0/2116394635 conn(0x5596165b7800 msgr2=0x559616521800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 -- 172.21.15.123:0/2116394635 shutdown_connections 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 -- 172.21.15.123:0/2116394635 wait complete. 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 Processor -- start 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.117+0000 7f5b9a558280 1 -- start start 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7800 0x5596171fa100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559617292000 0x5596171fa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b9a558280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b9a558280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5596164803c0 con 0x559617292400 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b9a558280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x559616480000 con 0x559617292000 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b9a558280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x559616481180 con 0x5596165b7800 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b93208700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b93208700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33760/0 (socket says 172.21.15.123:33760) 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b93208700 1 -- 172.21.15.123:0/1202170159 learned_addr learned my addr 172.21.15.123:0/1202170159 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b9420a700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7800 0x5596171fa100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.118+0000 7f5b93a09700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559617292000 0x5596171fa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b93208700 1 -- 172.21.15.123:0/1202170159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7800 msgr2=0x5596171fa100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b93208700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7800 0x5596171fa100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b93208700 1 -- 172.21.15.123:0/1202170159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559617292000 msgr2=0x5596171fa680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b93208700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x559617292000 0x5596171fa680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b93208700 1 -- 172.21.15.123:0/1202170159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559616464ea0 con 0x559617292400 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b93208700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fac00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559617229410 tx=0x55961724ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b8d9fd700 1 -- 172.21.15.123:0/1202170159 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5596164791e0 con 0x559617292400 2023-10-06T04:32:47.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b8d9fd700 1 -- 172.21.15.123:0/1202170159 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559617244780 con 0x559617292400 2023-10-06T04:32:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b8d9fd700 1 -- 172.21.15.123:0/1202170159 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x559616478dc0 con 0x559617292400 2023-10-06T04:32:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x559616465a00 con 0x559617292400 2023-10-06T04:32:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x559616465860 con 0x559617292400 2023-10-06T04:32:47.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.119+0000 7f5b9a558280 4 main: resolving pool 7 2023-10-06T04:32:47.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.120+0000 7f5b8d9fd700 1 -- 172.21.15.123:0/1202170159 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x5596171b7080 con 0x559617292400 2023-10-06T04:32:47.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b8d9fd700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5596172b0800 0x5596171fb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b8d9fd700 1 -- 172.21.15.123:0/1202170159 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x559616484a80 con 0x559617292400 2023-10-06T04:32:47.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b9a558280 4 main: creating IoCtx.. 2023-10-06T04:32:47.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b9420a700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5596172b0800 0x5596171fb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b9a558280 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x559617292800 0x5596171fb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:1 7.b 7:d0630e4c:::mds0_sessionmap:head [omap-clear,omap-set-header in=14b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x559617292c00 con 0x559617292800 2023-10-06T04:32:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b93a09700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x559617292800 0x5596171fb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b9420a700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5596172b0800 0x5596171fb700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5596171cd230 tx=0x55961724ec00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.121+0000 7f5b93a09700 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x559617292800 0x5596171fb180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.122+0000 7f5b93a09700 1 -- 172.21.15.123:0/1202170159 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== osd_op_reply(1 mds0_sessionmap [omap-clear,omap-set-header] v42'1 uv1 ondisk = 0) v8 ==== 201+0+0 (crc 0 0 0) 0x559617222fc0 con 0x559617292800 2023-10-06T04:32:47.131 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:47.132 INFO:teuthology.orchestra.run.smithi123.stdout: "0": { 2023-10-06T04:32:47.132 INFO:teuthology.orchestra.run.smithi123.stdout: "data": {}, 2023-10-06T04:32:47.132 INFO:teuthology.orchestra.run.smithi123.stdout: "result": 0 2023-10-06T04:32:47.132 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:47.132 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:47.132 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.122+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x559617292800 msgr2=0x5596171fb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.122+0000 7f5b9a558280 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x559617292800 0x5596171fb180 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5596172b0800 msgr2=0x5596171fb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5596172b0800 0x5596171fb700 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x5596171cd230 tx=0x55961724ec00 comp rx=0 tx=0).stop 2023-10-06T04:32:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 msgr2=0x5596171fac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 --2- 172.21.15.123:0/1202170159 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x559617292400 0x5596171fac00 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x559617229410 tx=0x55961724ecc0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b93a09700 1 -- 172.21.15.123:0/1202170159 reap_dead start 2023-10-06T04:32:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 shutdown_connections 2023-10-06T04:32:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 >> 172.21.15.123:0/1202170159 conn(0x5596165b7c00 msgr2=0x559616521800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 shutdown_connections 2023-10-06T04:32:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/1202170159 wait complete. 2023-10-06T04:32:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/2739569417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596165b6c00 msgr2=0x5596171f9080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596165b6c00 0x5596171f9080 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x5596171cdc50 tx=0x5596171d33e0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/2739569417 shutdown_connections 2023-10-06T04:32:47.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5596165b7000 0x5596171f9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5596165b6800 0x5596171f8b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 --2- 172.21.15.123:0/2739569417 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5596165b6c00 0x5596171f9080 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/2739569417 >> 172.21.15.123:0/2739569417 conn(0x5596165b6000 msgr2=0x559616521000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/2739569417 shutdown_connections 2023-10-06T04:32:47.138 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.123+0000 7f5b9a558280 1 -- 172.21.15.123:0/2739569417 wait complete. 2023-10-06T04:32:47.139 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'session'] in time 0:00:00.096892, result: { "0": { "data": {}, "result": 0 } } 2023-10-06T04:32:47.139 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset snap 2023-10-06T04:32:47.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 -- start start 2023-10-06T04:32:47.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 Processor -- start 2023-10-06T04:32:47.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827a400 0x5603b826a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b827a800 0x5603b826ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827ac00 0x5603b826b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fdcc0 con 0x5603b827a800 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fdb80 con 0x5603b827a400 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad29a02280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fde00 con 0x5603b827ac00 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad226b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b827a800 0x5603b826ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad226b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b827a800 0x5603b826ab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33772/0 (socket says 172.21.15.123:33772) 2023-10-06T04:32:47.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad226b2700 1 -- 172.21.15.123:0/2937751889 learned_addr learned my addr 172.21.15.123:0/2937751889 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad22eb3700 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827a400 0x5603b826a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.210+0000 7fad236b4700 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827ac00 0x5603b826b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad226b2700 1 -- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827ac00 msgr2=0x5603b826b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad226b2700 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827ac00 0x5603b826b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad226b2700 1 -- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827a400 msgr2=0x5603b826a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad226b2700 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827a400 0x5603b826a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad226b2700 1 -- 172.21.15.123:0/2937751889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603b74e0b60 con 0x5603b827a800 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad226b2700 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b827a800 0x5603b826ab00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5603b8249e00 tx=0x5603b824fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5bb04436b3dd418c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad206ae700 1 -- 172.21.15.123:0/2937751889 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5603b74f5600 con 0x5603b827a800 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad206ae700 1 -- 172.21.15.123:0/2937751889 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5603b7512900 con 0x5603b827a800 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad206ae700 1 -- 172.21.15.123:0/2937751889 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5603b74f51e0 con 0x5603b827a800 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad29a02280 1 -- 172.21.15.123:0/2937751889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603b74e0000 con 0x5603b827a800 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad29a02280 1 -- 172.21.15.123:0/2937751889 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5603b74e1860 con 0x5603b827a800 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad29a02280 4 waiting for MDS map... 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad206ae700 1 -- 172.21.15.123:0/2937751889 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x5603b7500c40 con 0x5603b827a800 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad206ae700 1 -- 172.21.15.123:0/2937751889 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x5603b8291200 con 0x5603b827a800 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad29a02280 4 Got MDS map 22 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad29a02280 10 main: main 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.211+0000 7fad29a02280 4 main: connecting to RADOS... 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.212+0000 7fad29a02280 1 Processor -- start 2023-10-06T04:32:47.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.212+0000 7fad29a02280 1 -- start start 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.212+0000 7fad29a02280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827b800 0x5603b826b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.212+0000 7fad29a02280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827bc00 0x5603b826bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad29a02280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b830e000 0x5603b826c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad29a02280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fd540 con 0x5603b830e000 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad29a02280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fdb80 con 0x5603b827bc00 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad29a02280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fdcc0 con 0x5603b827b800 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b830e000 0x5603b826c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b830e000 0x5603b826c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33788/0 (socket says 172.21.15.123:33788) 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 -- 172.21.15.123:0/751826568 learned_addr learned my addr 172.21.15.123:0/751826568 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad236b4700 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827b800 0x5603b826b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad22eb3700 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827bc00 0x5603b826bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 -- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827b800 msgr2=0x5603b826b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827b800 0x5603b826b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 -- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827bc00 msgr2=0x5603b826bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827bc00 0x5603b826bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 -- 172.21.15.123:0/751826568 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603b74e1a00 con 0x5603b830e000 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad22eb3700 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827bc00 0x5603b826bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad226b2700 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b830e000 0x5603b826c100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5603b82b47b0 tx=0x5603b824eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56852fe42a0324e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad1e6aa700 1 -- 172.21.15.123:0/751826568 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5603b74f5600 con 0x5603b830e000 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad1e6aa700 1 -- 172.21.15.123:0/751826568 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5603b82bc600 con 0x5603b830e000 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.213+0000 7fad1e6aa700 1 -- 172.21.15.123:0/751826568 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5603b74f51e0 con 0x5603b830e000 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 -- 172.21.15.123:0/751826568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b830e000 msgr2=0x5603b826c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b830e000 0x5603b826c100 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x5603b82b47b0 tx=0x5603b824eae0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 -- 172.21.15.123:0/751826568 shutdown_connections 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b830e000 0x5603b826c100 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827b800 0x5603b826b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 --2- 172.21.15.123:0/751826568 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827bc00 0x5603b826bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 -- 172.21.15.123:0/751826568 >> 172.21.15.123:0/751826568 conn(0x5603b827b000 msgr2=0x5603b759d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 -- 172.21.15.123:0/751826568 shutdown_connections 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 -- 172.21.15.123:0/751826568 wait complete. 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 Processor -- start 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.214+0000 7fad29a02280 1 -- start start 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827bc00 0x5603b826cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827b800 0x5603b826c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.224 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b8330000 0x5603b826c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fcdc0 con 0x5603b8330000 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fc3c0 con 0x5603b827b800 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5603b74fc000 con 0x5603b827bc00 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b8330000 0x5603b826c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b8330000 0x5603b826c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33798/0 (socket says 172.21.15.123:33798) 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 -- 172.21.15.123:0/1272110983 learned_addr learned my addr 172.21.15.123:0/1272110983 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad236b4700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827bc00 0x5603b826cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad22eb3700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827b800 0x5603b826c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 -- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827bc00 msgr2=0x5603b826cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827bc00 0x5603b826cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 -- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827b800 msgr2=0x5603b826c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.225 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827b800 0x5603b826c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 -- 172.21.15.123:0/1272110983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5603b74e1860 con 0x5603b8330000 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad226b2700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b8330000 0x5603b826c100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5603b833a030 tx=0x5603b8326300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad1cea7700 1 -- 172.21.15.123:0/1272110983 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5603b74f5080 con 0x5603b8330000 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad1cea7700 1 -- 172.21.15.123:0/1272110983 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5603b7512300 con 0x5603b8330000 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad1cea7700 1 -- 172.21.15.123:0/1272110983 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5603b74f5600 con 0x5603b8330000 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5603b74e1ba0 con 0x5603b8330000 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5603b74e09c0 con 0x5603b8330000 2023-10-06T04:32:47.226 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.215+0000 7fad29a02280 4 main: resolving pool 7 2023-10-06T04:32:47.227 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad1cea7700 1 -- 172.21.15.123:0/1272110983 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x5603b8233080 con 0x5603b8330000 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad1cea7700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5603b8336800 0x5603b826d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad1cea7700 1 -- 172.21.15.123:0/1272110983 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x5603b7500a80 con 0x5603b8330000 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad29a02280 4 main: creating IoCtx.. 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad236b4700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5603b8336800 0x5603b826d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad29a02280 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5603b8330400 0x5603b826bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:1 7.d 7:b50e409b:::mds_snaptable:head [writefull 0~70 in=70b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5603b8330800 con 0x5603b8330400 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad22eb3700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5603b8330400 0x5603b826bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.217+0000 7fad236b4700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5603b8336800 0x5603b826d700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5603b8249cb0 tx=0x5603b83262a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.218+0000 7fad22eb3700 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5603b8330400 0x5603b826bb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad22eb3700 1 -- 172.21.15.123:0/1272110983 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== osd_op_reply(1 mds_snaptable [writefull 0~70] v42'1 uv1 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5603b82a2fc0 con 0x5603b8330400 2023-10-06T04:32:47.229 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:47.229 INFO:teuthology.orchestra.run.smithi123.stdout: "result": 0 2023-10-06T04:32:47.229 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:47.229 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5603b8330400 msgr2=0x5603b826bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad29a02280 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5603b8330400 0x5603b826bb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5603b8336800 msgr2=0x5603b826d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad29a02280 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5603b8336800 0x5603b826d700 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x5603b8249cb0 tx=0x5603b83262a0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b8330000 msgr2=0x5603b826c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad29a02280 1 --2- 172.21.15.123:0/1272110983 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b8330000 0x5603b826c100 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x5603b833a030 tx=0x5603b8326300 comp rx=0 tx=0).stop 2023-10-06T04:32:47.231 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.219+0000 7fad22eb3700 1 -- 172.21.15.123:0/1272110983 reap_dead start 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 shutdown_connections 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 >> 172.21.15.123:0/1272110983 conn(0x5603b827b000 msgr2=0x5603b759d800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 shutdown_connections 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/1272110983 wait complete. 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/2937751889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b827a800 msgr2=0x5603b826ab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b827a800 0x5603b826ab00 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x5603b8249e00 tx=0x5603b824fa40 comp rx=0 tx=0).stop 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/2937751889 shutdown_connections 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5603b827ac00 0x5603b826b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5603b827a400 0x5603b826a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 --2- 172.21.15.123:0/2937751889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5603b827a800 0x5603b826ab00 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/2937751889 >> 172.21.15.123:0/2937751889 conn(0x5603b827a000 msgr2=0x5603b759d000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/2937751889 shutdown_connections 2023-10-06T04:32:47.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.220+0000 7fad29a02280 1 -- 172.21.15.123:0/2937751889 wait complete. 2023-10-06T04:32:47.233 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'snap'] in time 0:00:00.093994, result: { "result": 0 } 2023-10-06T04:32:47.233 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-table-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 recovery_fs:0 reset inode 2023-10-06T04:32:47.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.307+0000 7f23d4f21280 1 -- start start 2023-10-06T04:32:47.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.307+0000 7f23d4f21280 1 Processor -- start 2023-10-06T04:32:47.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2036400 0x5652c202a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c2036800 0x5652c202ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2036c00 0x5652c202b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b3cc0 con 0x5652c2036400 2023-10-06T04:32:47.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b3e00 con 0x5652c2036800 2023-10-06T04:32:47.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b3540 con 0x5652c2036c00 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2036400 0x5652c202a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2036400 0x5652c202a580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33806/0 (socket says 172.21.15.123:33806) 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 -- 172.21.15.123:0/4173549836 learned_addr learned my addr 172.21.15.123:0/4173549836 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23cebd3700 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2036c00 0x5652c202b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23cdbd1700 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c2036800 0x5652c202ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 -- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2036c00 msgr2=0x5652c202b080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2036c00 0x5652c202b080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 -- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c2036800 msgr2=0x5652c202ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c2036800 0x5652c202ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.320 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 -- 172.21.15.123:0/4173549836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c1296340 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23ce3d2700 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2036400 0x5652c202a580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5652c1fffda0 tx=0x5652c2054120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dd8d5f0b629a1f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23cbbcd700 1 -- 172.21.15.123:0/4173549836 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5652c12ab600 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23cbbcd700 1 -- 172.21.15.123:0/4173549836 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5652c12c8900 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23cbbcd700 1 -- 172.21.15.123:0/4173549836 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5652c12ab1e0 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 -- 172.21.15.123:0/4173549836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652c1296b60 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 1 -- 172.21.15.123:0/4173549836 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5652c1296ea0 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.308+0000 7f23d4f21280 4 waiting for MDS map... 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.309+0000 7f23cbbcd700 1 -- 172.21.15.123:0/4173549836 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x5652c12b6c40 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.309+0000 7f23cbbcd700 1 -- 172.21.15.123:0/4173549836 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x5652c204c600 con 0x5652c2036400 2023-10-06T04:32:47.321 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.309+0000 7f23d4f21280 4 Got MDS map 22 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.309+0000 7f23d4f21280 10 main: main 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.309+0000 7f23d4f21280 4 main: connecting to RADOS... 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.309+0000 7f23d4f21280 1 Processor -- start 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.309+0000 7f23d4f21280 1 -- start start 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037800 0x5652c202b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037c00 0x5652c202bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20d0000 0x5652c202c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23d4f21280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b3b80 con 0x5652c2037800 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23d4f21280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b3e00 con 0x5652c20d0000 2023-10-06T04:32:47.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23d4f21280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b2dc0 con 0x5652c2037c00 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037800 0x5652c202b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037800 0x5652c202b600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33820/0 (socket says 172.21.15.123:33820) 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 -- 172.21.15.123:0/3150986229 learned_addr learned my addr 172.21.15.123:0/3150986229 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23ce3d2700 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037c00 0x5652c202bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cdbd1700 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20d0000 0x5652c202c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 -- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037c00 msgr2=0x5652c202bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037c00 0x5652c202bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 -- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20d0000 msgr2=0x5652c202c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20d0000 0x5652c202c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.323 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 -- 172.21.15.123:0/3150986229 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c1296340 con 0x5652c2037800 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cdbd1700 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20d0000 0x5652c202c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23cebd3700 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037800 0x5652c202b600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x5652c2061290 tx=0x5652c207ec00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=475669e7572d218b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23c9bc9700 1 -- 172.21.15.123:0/3150986229 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5652c12ab600 con 0x5652c2037800 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23c9bc9700 1 -- 172.21.15.123:0/3150986229 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5652c12c8300 con 0x5652c2037800 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.310+0000 7f23c9bc9700 1 -- 172.21.15.123:0/3150986229 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5652c12ab1e0 con 0x5652c2037800 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 -- 172.21.15.123:0/3150986229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037800 msgr2=0x5652c202b600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037800 0x5652c202b600 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x5652c2061290 tx=0x5652c207ec00 comp rx=0 tx=0).stop 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 -- 172.21.15.123:0/3150986229 shutdown_connections 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037800 0x5652c202b600 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.324 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20d0000 0x5652c202c100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 --2- 172.21.15.123:0/3150986229 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037c00 0x5652c202bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 -- 172.21.15.123:0/3150986229 >> 172.21.15.123:0/3150986229 conn(0x5652c2037000 msgr2=0x5652c1353800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 -- 172.21.15.123:0/3150986229 shutdown_connections 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 -- 172.21.15.123:0/3150986229 wait complete. 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.311+0000 7f23d4f21280 1 Processor -- start 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23d4f21280 1 -- start start 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037c00 0x5652c202cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037800 0x5652c202c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23d4f21280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20e8000 0x5652c202c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23d4f21280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b23c0 con 0x5652c2037c00 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23d4f21280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b2000 con 0x5652c20e8000 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23d4f21280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5652c12b3180 con 0x5652c2037800 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037c00 0x5652c202cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037c00 0x5652c202cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33822/0 (socket says 172.21.15.123:33822) 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 -- 172.21.15.123:0/610845513 learned_addr learned my addr 172.21.15.123:0/610845513 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23ce3d2700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037800 0x5652c202c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cdbd1700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20e8000 0x5652c202c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 -- 172.21.15.123:0/610845513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037800 msgr2=0x5652c202c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2037800 0x5652c202c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 -- 172.21.15.123:0/610845513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20e8000 msgr2=0x5652c202c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c20e8000 0x5652c202c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.312+0000 7f23cebd3700 1 -- 172.21.15.123:0/610845513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5652c1297860 con 0x5652c2037c00 2023-10-06T04:32:47.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.313+0000 7f23cebd3700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037c00 0x5652c202cc00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5652c2061f50 tx=0x5652c207ecc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.313+0000 7f23c83c6700 1 -- 172.21.15.123:0/610845513 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5652c12aadc0 con 0x5652c2037c00 2023-10-06T04:32:47.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.313+0000 7f23c83c6700 1 -- 172.21.15.123:0/610845513 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5652c12c8c00 con 0x5652c2037c00 2023-10-06T04:32:47.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.313+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5652c1296b60 con 0x5652c2037c00 2023-10-06T04:32:47.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.313+0000 7f23c83c6700 1 -- 172.21.15.123:0/610845513 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5652c12ab600 con 0x5652c2037c00 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.313+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5652c1296ea0 con 0x5652c2037c00 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.313+0000 7f23d4f21280 4 main: resolving pool 7 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.314+0000 7f23c83c6700 1 -- 172.21.15.123:0/610845513 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x5652c1fe9080 con 0x5652c2037c00 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.314+0000 7f23c83c6700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5652c20ee400 0x5652c202d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.314+0000 7f23c83c6700 1 -- 172.21.15.123:0/610845513 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x5652c12b6a80 con 0x5652c2037c00 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.314+0000 7f23ce3d2700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5652c20ee400 0x5652c202d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.314+0000 7f23d4f21280 4 main: creating IoCtx.. 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.314+0000 7f23d4f21280 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5652c20e8400 0x5652c202bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.314+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:1 7.13 7:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5652c20e8800 con 0x5652c20e8400 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.315+0000 7f23cdbd1700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5652c20e8400 0x5652c202bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.315+0000 7f23ce3d2700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5652c20ee400 0x5652c202d700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5652c20605a0 tx=0x5652c20058c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.315+0000 7f23cdbd1700 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5652c20e8400 0x5652c202bb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23cdbd1700 1 -- 172.21.15.123:0/610845513 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(1 mds0_inotable [writefull 0~34] v42'2 uv2 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x5652c204afc0 con 0x5652c20e8400 2023-10-06T04:32:47.329 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: "0": { 2023-10-06T04:32:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: "data": {}, 2023-10-06T04:32:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: "result": 0 2023-10-06T04:32:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:47.330 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:47.330 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:47.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5652c20e8400 msgr2=0x5652c202bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.332 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23d4f21280 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5652c20e8400 0x5652c202bb80 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5652c20ee400 msgr2=0x5652c202d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23d4f21280 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5652c20ee400 0x5652c202d700 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x5652c20605a0 tx=0x5652c20058c0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037c00 msgr2=0x5652c202cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23d4f21280 1 --2- 172.21.15.123:0/610845513 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2037c00 0x5652c202cc00 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x5652c2061f50 tx=0x5652c207ecc0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23cdbd1700 1 -- 172.21.15.123:0/610845513 reap_dead start 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.316+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 shutdown_connections 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 >> 172.21.15.123:0/610845513 conn(0x5652c2037000 msgr2=0x5652c1353800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 shutdown_connections 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/610845513 wait complete. 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/4173549836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2036400 msgr2=0x5652c202a580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2036400 0x5652c202a580 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x5652c1fffda0 tx=0x5652c2054120 comp rx=0 tx=0).stop 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/4173549836 shutdown_connections 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5652c2036400 0x5652c202a580 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5652c2036c00 0x5652c202b080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 --2- 172.21.15.123:0/4173549836 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5652c2036800 0x5652c202ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/4173549836 >> 172.21.15.123:0/4173549836 conn(0x5652c2036000 msgr2=0x5652c1353000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/4173549836 shutdown_connections 2023-10-06T04:32:47.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.317+0000 7f23d4f21280 1 -- 172.21.15.123:0/4173549836 wait complete. 2023-10-06T04:32:47.335 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-table-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'recovery_fs:0', 'reset', 'inode'] in time 0:00:00.102054, result: { "0": { "data": {}, "result": 0 } } 2023-10-06T04:32:47.335 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_extents --alternate-pool recovery_fs_metadata --filesystem cephfs cephfs_data 2023-10-06T04:32:47.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 -- start start 2023-10-06T04:32:47.415 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 Processor -- start 2023-10-06T04:32:47.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434952400 0x55c43558a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434952800 0x55c43558ab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434952c00 0x55c43558b080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43481dcc0 con 0x55c434952c00 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43481de00 con 0x55c434952400 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c43481d540 con 0x55c434952800 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6bfc27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434952c00 0x55c43558b080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6bfc27700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434952c00 0x55c43558b080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33838/0 (socket says 172.21.15.123:33838) 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6bfc27700 1 -- 172.21.15.123:0/1596361384 learned_addr learned my addr 172.21.15.123:0/1596361384 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.413+0000 7fe6bf426700 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434952400 0x55c43558a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bec25700 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434952800 0x55c43558ab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bfc27700 1 -- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434952800 msgr2=0x55c43558ab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434952800 0x55c43558ab00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bfc27700 1 -- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434952400 msgr2=0x55c43558a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434952400 0x55c43558a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bfc27700 1 -- 172.21.15.123:0/1596361384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c434800340 con 0x55c434952c00 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434952c00 0x55c43558b080 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55c4355ac750 tx=0x55c43556fa40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=222bc58746bbd45f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bcc21700 1 -- 172.21.15.123:0/1596361384 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c434815080 con 0x55c434952c00 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bcc21700 1 -- 172.21.15.123:0/1596361384 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c434832f00 con 0x55c434952c00 2023-10-06T04:32:47.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bcc21700 1 -- 172.21.15.123:0/1596361384 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c434815600 con 0x55c434952c00 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6c61d5740 1 -- 172.21.15.123:0/1596361384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c434800000 con 0x55c434952c00 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6c61d5740 1 -- 172.21.15.123:0/1596361384 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55c434800ea0 con 0x55c434952c00 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6c61d5740 4 waiting for MDS map... 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bcc21700 1 -- 172.21.15.123:0/1596361384 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55c434820c40 con 0x55c434952c00 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6bcc21700 1 -- 172.21.15.123:0/1596361384 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x55c4355bf200 con 0x55c434952c00 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.414+0000 7fe6c61d5740 4 Got MDS map 22 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.415+0000 7fe6c61d5740 4 datascan.main: Using metadata pool output 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.415+0000 7fe6c61d5740 4 datascan.main: connecting to RADOS... 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.415+0000 7fe6c61d5740 1 Processor -- start 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.415+0000 7fe6c61d5740 1 -- start start 2023-10-06T04:32:47.426 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434953800 0x55c43558b600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 0x55c43558bb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c435630000 0x55c43558c100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43481db80 con 0x55c435630000 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43481de00 con 0x55c434953800 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c43481dcc0 con 0x55c434953c00 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c435630000 0x55c43558c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c435630000 0x55c43558c100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33854/0 (socket says 172.21.15.123:33854) 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 -- 172.21.15.123:0/3500856186 learned_addr learned my addr 172.21.15.123:0/3500856186 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bf426700 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 0x55c43558bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434953800 0x55c43558b600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 -- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 msgr2=0x55c43558bb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 0x55c43558bb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 -- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434953800 msgr2=0x55c43558b600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434953800 0x55c43558b600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 -- 172.21.15.123:0/3500856186 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c434801860 con 0x55c435630000 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bec25700 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c435630000 0x55c43558c100 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55c4355ad5f0 tx=0x55c43564a0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e3dfe623752a7b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bac1d700 1 -- 172.21.15.123:0/3500856186 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c434815080 con 0x55c435630000 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bac1d700 1 -- 172.21.15.123:0/3500856186 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c434832600 con 0x55c435630000 2023-10-06T04:32:47.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.416+0000 7fe6bac1d700 1 -- 172.21.15.123:0/3500856186 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c434815600 con 0x55c435630000 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 -- 172.21.15.123:0/3500856186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c435630000 msgr2=0x55c43558c100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c435630000 0x55c43558c100 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x55c4355ad5f0 tx=0x55c43564a0c0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 -- 172.21.15.123:0/3500856186 shutdown_connections 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c435630000 0x55c43558c100 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434953800 0x55c43558b600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/3500856186 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 0x55c43558bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 -- 172.21.15.123:0/3500856186 >> 172.21.15.123:0/3500856186 conn(0x55c434953000 msgr2=0x55c4348bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 -- 172.21.15.123:0/3500856186 shutdown_connections 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 -- 172.21.15.123:0/3500856186 wait complete. 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 Processor -- start 2023-10-06T04:32:47.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.417+0000 7fe6c61d5740 1 -- start start 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 0x55c43558cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434953800 0x55c43558c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6c61d5740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c435666000 0x55c43558c100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43481cdc0 con 0x55c434953800 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c43481c3c0 con 0x55c435666000 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6c61d5740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c43481c000 con 0x55c434953c00 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434953800 0x55c43558c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434953800 0x55c43558c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33860/0 (socket says 172.21.15.123:33860) 2023-10-06T04:32:47.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 learned_addr learned my addr 172.21.15.123:0/2001634380 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 0x55c43558cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c435666000 0x55c43558c100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 msgr2=0x55c43558cc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434953c00 0x55c43558cc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c435666000 msgr2=0x55c43558c100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c435666000 0x55c43558c100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c4348009c0 con 0x55c434953800 2023-10-06T04:32:47.432 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6bf426700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434953800 0x55c43558c680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c4355f46c0 tx=0x55c43564a120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6b941a700 1 -- 172.21.15.123:0/2001634380 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c434815080 con 0x55c434953800 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6b941a700 1 -- 172.21.15.123:0/2001634380 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c434832000 con 0x55c434953800 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6b941a700 1 -- 172.21.15.123:0/2001634380 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55c434815600 con 0x55c434953800 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.418+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55c434801a00 con 0x55c434953800 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.419+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c4348009c0 con 0x55c434953800 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.419+0000 7fe6c61d5740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6b941a700 1 -- 172.21.15.123:0/2001634380 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55c435553080 con 0x55c434953800 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6b941a700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55c43566c400 0x55c43558d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6b941a700 1 -- 172.21.15.123:0/2001634380 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55c434820a80 con 0x55c434953800 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55c43566c400 0x55c43558d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6c61d5740 4 datascan.main: data pool 'cephfs_data' has ID 6 2023-10-06T04:32:47.433 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6c61d5740 4 datascan.main: opening data pool 'cephfs_data' 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55c435666400 0x55c43558bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:1 6.0 6:00000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435666800 con 0x55c435666400 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.420+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55c435666400 0x55c43558bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.421+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55c43566c400 0x55c43558d700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55c4355ad710 tx=0x55c4355c00c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.421+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55c435666400 0x55c43558bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.421+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(1 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.421+0000 7fe6ba41c700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55c43566c800 0x55c435762000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.421+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:2 6.20 6:04000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566cc00 con 0x55c43566c800 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.421+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55c43566c800 0x55c435762000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.434 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.421+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55c43566c800 0x55c435762000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.422+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== osd_op_reply(2 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c43566c800 2023-10-06T04:32:47.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.422+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:3 6.10 6:08000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566d400 con 0x55c435666400 2023-10-06T04:32:47.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.422+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== osd_op_reply(3 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.422+0000 7fe6ba41c700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55c43566d000 0x55c435762580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.422+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:4 6.30 6:0c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566d400 con 0x55c43566d000 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.422+0000 7fe6bf426700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55c43566d000 0x55c435762580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.423+0000 7fe6bf426700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55c43566d000 0x55c435762580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.423+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 1 ==== osd_op_reply(4 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.423+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:5 6.8 6:10000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566dc00 con 0x55c43566d000 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.423+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 2 ==== osd_op_reply(5 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.423+0000 7fe6ba41c700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55c43566d800 0x55c435762b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.423+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:6 6.28 6:14000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435666800 con 0x55c43566d800 2023-10-06T04:32:47.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.423+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55c43566d800 0x55c435762b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.424+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55c43566d800 0x55c435762b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.424+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.2 v2:172.21.15.123:6824/119723 1 ==== osd_op_reply(6 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d800 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.424+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:7 6.18 6:18000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774000 con 0x55c435666400 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.424+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 3 ==== osd_op_reply(7 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.424+0000 7fe6ba41c700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55c435774c00 0x55c435763080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.424+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:8 6.38 6:1c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435775000 con 0x55c435774c00 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.424+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55c435774c00 0x55c435763080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.425+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55c435774c00 0x55c435763080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.425+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== osd_op_reply(8 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.425+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:9 6.4 6:20000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774400 con 0x55c43566c800 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.425+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.1 v2:172.21.15.123:6816/119720 2 ==== osd_op_reply(9 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c43566c800 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.425+0000 7fe6ba41c700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55c435775400 0x55c435763600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.425+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:10 6.24 6:24000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435775800 con 0x55c435775400 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.425+0000 7fe6bf426700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55c435775400 0x55c435763600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.426+0000 7fe6bf426700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55c435775400 0x55c435763600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.426+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 1 ==== osd_op_reply(10 [pgnls start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.426+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:11 6.14 6:28000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774800 con 0x55c43566d800 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.426+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.2 v2:172.21.15.123:6824/119723 2 ==== osd_op_reply(11 [pgnls start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d800 2023-10-06T04:32:47.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.426+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:12 6.34 6:2c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774000 con 0x55c435666400 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.426+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 4 ==== osd_op_reply(12 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.426+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:13 6.c 6:30000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774800 con 0x55c435774c00 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.427+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== osd_op_reply(13 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.427+0000 7fe6ba41c700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55c43566d400 0x55c435763b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.427+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:14 6.2c 6:34000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566dc00 con 0x55c43566d400 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.427+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55c43566d400 0x55c435763b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.427+0000 7fe6bec25700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55c43566d400 0x55c435763b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== osd_op_reply(14 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d400 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:15 6.1c 6:38000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435775800 con 0x55c43566d000 2023-10-06T04:32:47.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 3 ==== osd_op_reply(15 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:16 6.3c 6:3c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566cc00 con 0x55c43566d800 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.2 v2:172.21.15.123:6824/119723 3 ==== osd_op_reply(16 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d800 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6bac1d700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55c435775000 0x55c435764100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:17 6.2 6:40000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774400 con 0x55c435775000 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.428+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55c435775000 0x55c435764100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.429+0000 7fe6bfc27700 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55c435775000 0x55c435764100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.429+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.4 v2:172.21.15.177:6808/119946 1 ==== osd_op_reply(17 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435775000 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.429+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:18 6.22 6:44000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435666800 con 0x55c43566d400 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.429+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.6 v2:172.21.15.177:6816/119976 2 ==== osd_op_reply(18 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d400 2023-10-06T04:32:47.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.429+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:19 6.12 6:48000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774000 con 0x55c43566d000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 4 ==== osd_op_reply(19 [pgnls start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:20 6.32 6:4c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c000 con 0x55c435775000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.4 v2:172.21.15.177:6808/119946 2 ==== osd_op_reply(20 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435775000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:21 6.a 6:50000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435775c00 con 0x55c435775400 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 2 ==== osd_op_reply(21 [pgnls start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:22 6.2a 6:54000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c400 con 0x55c435775000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.4 v2:172.21.15.177:6808/119946 3 ==== osd_op_reply(22 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435775000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.430+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:23 6.1a 6:58000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435666800 con 0x55c435775000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.4 v2:172.21.15.177:6808/119946 4 ==== osd_op_reply(23 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435775000 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:24 6.3a 6:5c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c800 con 0x55c43566c800 2023-10-06T04:32:47.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.1 v2:172.21.15.123:6816/119720 3 ==== osd_op_reply(24 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c43566c800 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:25 6.6 6:60000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566dc00 con 0x55c435774c00 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 3 ==== osd_op_reply(25 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:26 6.26 6:64000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578cc00 con 0x55c43566c800 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.1 v2:172.21.15.123:6816/119720 4 ==== osd_op_reply(26 [pgnls start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c43566c800 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.431+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:27 6.16 6:68000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566cc00 con 0x55c435666400 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.432+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 5 ==== osd_op_reply(27 [pgnls start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.432+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:28 6.36 6:6c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578d000 con 0x55c435775400 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.432+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 3 ==== osd_op_reply(28 [pgnls start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.432+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:29 6.e 6:70000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578d000 con 0x55c435775000 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.432+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.4 v2:172.21.15.177:6808/119946 5 ==== osd_op_reply(29 [pgnls start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435775000 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.432+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:30 6.2e 6:74000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578d400 con 0x55c43566d400 2023-10-06T04:32:47.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.6 v2:172.21.15.177:6816/119976 3 ==== osd_op_reply(30 [pgnls start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d400 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:31 6.1e 6:78000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435775800 con 0x55c435775000 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.4 v2:172.21.15.177:6808/119946 6 ==== osd_op_reply(31 [pgnls start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435775000 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:32 6.3e 6:7c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c800 con 0x55c435774c00 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 4 ==== osd_op_reply(32 [pgnls start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:33 6.1 6:80000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774000 con 0x55c43566c800 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.1 v2:172.21.15.123:6816/119720 5 ==== osd_op_reply(33 [pgnls start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c43566c800 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.433+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:34 6.21 6:84000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578cc00 con 0x55c43566d000 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 5 ==== osd_op_reply(34 [pgnls start_epoch 42 out=49b] v34'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:35 6.11 6:88000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435775c00 con 0x55c435774c00 2023-10-06T04:32:47.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 5 ==== osd_op_reply(35 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:36 6.31 6:8c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578d000 con 0x55c435775400 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 4 ==== osd_op_reply(36 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:37 6.9 6:90000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774800 con 0x55c435666400 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 6 ==== osd_op_reply(37 [pgnls start_epoch 42 out=49b] v34'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.434+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:38 6.29 6:94000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c000 con 0x55c43566d000 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.435+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 6 ==== osd_op_reply(38 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.435+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:39 6.19 6:98000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774400 con 0x55c435775400 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.435+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 5 ==== osd_op_reply(39 [pgnls start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.435+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:40 6.39 6:9c000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c400 con 0x55c435775400 2023-10-06T04:32:47.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.435+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 6 ==== osd_op_reply(40 [pgnls start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.435+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:41 6.5 6:a0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578cc00 con 0x55c43566d000 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.436+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 7 ==== osd_op_reply(41 [pgnls start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.436+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:42 6.25 6:a4000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566dc00 con 0x55c43566d400 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.436+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.6 v2:172.21.15.177:6816/119976 4 ==== osd_op_reply(42 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d400 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.436+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:43 6.15 6:a8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578d000 con 0x55c43566d000 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.436+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 8 ==== osd_op_reply(43 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.436+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:44 6.35 6:ac000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435666800 con 0x55c435775400 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 7 ==== osd_op_reply(44 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:45 6.d 6:b0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c000 con 0x55c435775400 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 8 ==== osd_op_reply(45 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:46 6.2d 6:b4000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435790000 con 0x55c43566d800 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.2 v2:172.21.15.123:6824/119723 4 ==== osd_op_reply(46 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d800 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:47 6.1d 6:b8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578c400 con 0x55c43566c800 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.1 v2:172.21.15.123:6816/119720 6 ==== osd_op_reply(47 [pgnls start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c43566c800 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:48 6.3d 6:bc000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435790400 con 0x55c435666400 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.437+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 7 ==== osd_op_reply(48 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.438+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:49 6.3 6:c0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578d800 con 0x55c43566d000 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.438+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 9 ==== osd_op_reply(49 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.438+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:50 6.23 6:c4000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435790800 con 0x55c43566d400 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.438+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.6 v2:172.21.15.177:6816/119976 5 ==== osd_op_reply(50 [pgnls start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d400 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.438+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:51 6.13 6:c8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43578dc00 con 0x55c435774c00 2023-10-06T04:32:47.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.438+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 6 ==== osd_op_reply(51 [pgnls start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.438+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:52 6.33 6:cc000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435790c00 con 0x55c435775400 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 9 ==== osd_op_reply(52 [pgnls start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:53 6.b 6:d0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435774400 con 0x55c435774c00 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 7 ==== osd_op_reply(53 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:54 6.2b 6:d4000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435791000 con 0x55c435775400 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 10 ==== osd_op_reply(54 [pgnls start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:55 6.1b 6:d8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435790c00 con 0x55c435774c00 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 8 ==== osd_op_reply(55 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.439+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:56 6.3b 6:dc000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435791400 con 0x55c43566d800 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.440+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.2 v2:172.21.15.123:6824/119723 5 ==== osd_op_reply(56 [pgnls start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d800 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.440+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:57 6.7 6:e0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566cc00 con 0x55c435775400 2023-10-06T04:32:47.464 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.440+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 11 ==== osd_op_reply(57 [pgnls start_epoch 42 out=81b] v34'8 uv8 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.440+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:58 6.7 6:e2e07930:::10000000001.00000000:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435666c00 con 0x55c435775400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.440+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 12 ==== osd_op_reply(58 10000000001.00000000 [stat out=16b] v0'0 uv8 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.440+0000 7fe6c61d5740 10 datascan.operator(): handling object 1099511627777.0 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.440+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:59 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435667000 con 0x55c435775400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.442+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 13 ==== osd_op_reply(59 10000000001.00000000 [call] v42'9 uv9 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.442+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:60 6.27 6:e4000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435667400 con 0x55c43566d400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.442+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.6 v2:172.21.15.177:6816/119976 6 ==== osd_op_reply(60 [pgnls start_epoch 42 out=49b] v34'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.442+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:61 6.17 6:e8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c43566dc00 con 0x55c435775000 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.442+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.4 v2:172.21.15.177:6808/119946 7 ==== osd_op_reply(61 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435775000 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.442+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:62 6.37 6:ec000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435791800 con 0x55c435666400 2023-10-06T04:32:47.465 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.0 v2:172.21.15.123:6808/119718 8 ==== osd_op_reply(62 [pgnls start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c435666400 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6bac1d700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:63 6.f 6:f0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435775800 con 0x55c43566d800 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6bec25700 1 -- 172.21.15.123:0/2001634380 <== osd.2 v2:172.21.15.123:6824/119723 6 ==== osd_op_reply(63 [pgnls start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355aafc0 con 0x55c43566d800 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:64 6.2f 6:f4000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435791c00 con 0x55c43566d000 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 10 ==== osd_op_reply(64 [pgnls start_epoch 42 out=81b] v34'9 uv1 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:65 6.2f 6:f526278d:::10000000001.00000001:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435667800 con 0x55c43566d000 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.7 v2:172.21.15.177:6824/119977 11 ==== osd_op_reply(65 10000000001.00000001 [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+16 (crc 0 0 0) 0x55c4355ab440 con 0x55c43566d000 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.443+0000 7fe6c61d5740 10 datascan.operator(): handling object 1099511627777.1 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.444+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:66 6.7 6:e2e07930:::10000000001.00000000:head [call cephfs.accumulate_inode_metadata in=112b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435667c00 con 0x55c435775400 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 <== osd.5 v2:172.21.15.177:6800/119947 14 ==== osd_op_reply(66 10000000001.00000000 [call] v42'10 uv10 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55c4355ab440 con 0x55c435775400 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:67 6.1f 6:f8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435791000 con 0x55c435774c00 2023-10-06T04:32:47.466 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 9 ==== osd_op_reply(67 [pgnls start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6ba41c700 1 -- 172.21.15.123:0/2001634380 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:68 6.3f 6:fc000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55c435791400 con 0x55c435774c00 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6bfc27700 1 -- 172.21.15.123:0/2001634380 <== osd.3 v2:172.21.15.123:6800/119719 10 ==== osd_op_reply(68 [pgnls start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55c4355ab200 con 0x55c435774c00 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55c435666400 msgr2=0x55c43558bb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55c435666400 0x55c43558bb80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55c43566c800 msgr2=0x55c435762000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55c43566c800 0x55c435762000 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55c43566d800 msgr2=0x55c435762b00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55c43566d800 0x55c435762b00 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55c435774c00 msgr2=0x55c435763080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55c435774c00 0x55c435763080 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 reap_dead start 2023-10-06T04:32:47.467 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55c435775000 msgr2=0x55c435764100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55c435775000 0x55c435764100 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55c435775400 msgr2=0x55c435763600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 reap_dead start 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55c435775400 0x55c435763600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55c43566d400 msgr2=0x55c435763b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55c43566d400 0x55c435763b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55c43566d000 msgr2=0x55c435762580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55c43566d000 0x55c435762580 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55c43566c400 msgr2=0x55c43558d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55c43566c400 0x55c43558d700 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x55c4355ad710 tx=0x55c4355c00c0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.468 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434953800 msgr2=0x55c43558c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/2001634380 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434953800 0x55c43558c680 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x55c4355f46c0 tx=0x55c43564a120 comp rx=0 tx=0).stop 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6bf426700 1 -- 172.21.15.123:0/2001634380 reap_dead start 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 shutdown_connections 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.445+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 >> 172.21.15.123:0/2001634380 conn(0x55c434953000 msgr2=0x55c4348bd800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 shutdown_connections 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 -- 172.21.15.123:0/2001634380 wait complete. 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 -- 172.21.15.123:0/1596361384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434952c00 msgr2=0x55c43558b080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434952c00 0x55c43558b080 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x55c4355ac750 tx=0x55c43556fa40 comp rx=0 tx=0).stop 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 -- 172.21.15.123:0/1596361384 shutdown_connections 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55c434952c00 0x55c43558b080 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55c434952400 0x55c43558a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.469 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 --2- 172.21.15.123:0/1596361384 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55c434952800 0x55c43558ab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 -- 172.21.15.123:0/1596361384 >> 172.21.15.123:0/1596361384 conn(0x55c434952000 msgr2=0x55c4348bd000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 -- 172.21.15.123:0/1596361384 shutdown_connections 2023-10-06T04:32:47.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.446+0000 7fe6c61d5740 1 -- 172.21.15.123:0/1596361384 wait complete. 2023-10-06T04:32:47.470 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_extents', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', 'cephfs_data'] in time 0:00:00.134816, result: 2023-10-06T04:32:47.470 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_inodes --alternate-pool recovery_fs_metadata --filesystem cephfs --force-corrupt --force-init cephfs_data 2023-10-06T04:32:47.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.533+0000 7f2f96d6b740 1 -- start start 2023-10-06T04:32:47.536 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.533+0000 7f2f96d6b740 1 Processor -- start 2023-10-06T04:32:47.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f0400 0x55a20ad36580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f0800 0x55a20ad36b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20a0f0c00 0x55a20ad37080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a209fbbcc0 con 0x55a20a0f0c00 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a209fbb540 con 0x55a20a0f0800 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a209fbbe00 con 0x55a20a0f0400 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f907bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20a0f0c00 0x55a20ad37080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f907bd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20a0f0c00 0x55a20ad37080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33888/0 (socket says 172.21.15.123:33888) 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f907bd700 1 -- 172.21.15.123:0/869629429 learned_addr learned my addr 172.21.15.123:0/869629429 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f0400 0x55a20ad36580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.534+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f0800 0x55a20ad36b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f907bd700 1 -- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f0400 msgr2=0x55a20ad36580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f907bd700 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f0400 0x55a20ad36580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f907bd700 1 -- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f0800 msgr2=0x55a20ad36b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f907bd700 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f0800 0x55a20ad36b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f907bd700 1 -- 172.21.15.123:0/869629429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a209f9e340 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f907bd700 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20a0f0c00 0x55a20ad37080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a20ad586c0 tx=0x55a20ad0d2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad162e1c622539e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f8d7b7700 1 -- 172.21.15.123:0/869629429 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a209fb3600 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f8d7b7700 1 -- 172.21.15.123:0/869629429 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a209fd0f00 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f8d7b7700 1 -- 172.21.15.123:0/869629429 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a209fb31e0 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f96d6b740 1 -- 172.21.15.123:0/869629429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a209f9e000 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f96d6b740 1 -- 172.21.15.123:0/869629429 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55a209f9eea0 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f96d6b740 4 waiting for MDS map... 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f8d7b7700 1 -- 172.21.15.123:0/869629429 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55a209fbec40 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f8d7b7700 1 -- 172.21.15.123:0/869629429 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x55a20ad48600 con 0x55a20a0f0c00 2023-10-06T04:32:47.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f96d6b740 4 Got MDS map 22 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f96d6b740 4 datascan.main: Using metadata pool output 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.535+0000 7f2f96d6b740 4 datascan.main: connecting to RADOS... 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 Processor -- start 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 -- start start 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 0x55a20ad37600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 0x55a20ad37b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20adcc000 0x55a20ad38100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a209fbbb80 con 0x55a20adcc000 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a209fbb540 con 0x55a20a0f1c00 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a209fbadc0 con 0x55a20a0f1800 2023-10-06T04:32:47.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.536+0000 7f2f8f7bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20adcc000 0x55a20ad38100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20adcc000 0x55a20ad38100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33890/0 (socket says 172.21.15.123:33890) 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/764924745 learned_addr learned my addr 172.21.15.123:0/764924745 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 0x55a20ad37b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 0x55a20ad37600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 msgr2=0x55a20ad37600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 0x55a20ad37600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 msgr2=0x55a20ad37b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 0x55a20ad37b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/764924745 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a209f9f860 con 0x55a20adcc000 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20adcc000 0x55a20ad38100 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55a20ad59650 tx=0x55a20ad42060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81614844ae1c418e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/764924745 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a209fb3080 con 0x55a20adcc000 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/764924745 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a209fd0000 con 0x55a20adcc000 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/764924745 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a209fb3600 con 0x55a20adcc000 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f96d6b740 1 -- 172.21.15.123:0/764924745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20adcc000 msgr2=0x55a20ad38100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20adcc000 0x55a20ad38100 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x55a20ad59650 tx=0x55a20ad42060 comp rx=0 tx=0).stop 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.537+0000 7f2f96d6b740 1 -- 172.21.15.123:0/764924745 shutdown_connections 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20adcc000 0x55a20ad38100 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 0x55a20ad37600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/764924745 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 0x55a20ad37b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 -- 172.21.15.123:0/764924745 >> 172.21.15.123:0/764924745 conn(0x55a20a0f1000 msgr2=0x55a20a05b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 -- 172.21.15.123:0/764924745 shutdown_connections 2023-10-06T04:32:47.602 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 -- 172.21.15.123:0/764924745 wait complete. 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 Processor -- start 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.538+0000 7f2f96d6b740 1 -- start start 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 0x55a20ad38c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 0x55a20ad38680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20add0000 0x55a20ad38100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a209fba3c0 con 0x55a20add0000 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a209fba000 con 0x55a20a0f1c00 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a209fbb180 con 0x55a20a0f1800 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20add0000 0x55a20ad38100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20add0000 0x55a20ad38100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33892/0 (socket says 172.21.15.123:33892) 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 learned_addr learned my addr 172.21.15.123:0/2201717452 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 0x55a20ad38c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 0x55a20ad38680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 msgr2=0x55a20ad38680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f1800 0x55a20ad38680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 msgr2=0x55a20ad38c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f1c00 0x55a20ad38c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a209f9e9c0 con 0x55a20add0000 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20add0000 0x55a20ad38100 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55a20ad598c0 tx=0x55a20ad42180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f89fb0700 1 -- 172.21.15.123:0/2201717452 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a209fb31e0 con 0x55a20add0000 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f89fb0700 1 -- 172.21.15.123:0/2201717452 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a209fd0300 con 0x55a20add0000 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55a209f9fa00 con 0x55a20add0000 2023-10-06T04:32:47.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f89fb0700 1 -- 172.21.15.123:0/2201717452 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55a209fb2dc0 con 0x55a20add0000 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a209f9f860 con 0x55a20add0000 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.539+0000 7f2f96d6b740 4 datascan.init: forcing metadata pool 'recovery_fs_metadata' 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f89fb0700 1 -- 172.21.15.123:0/2201717452 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55a20acf1080 con 0x55a20add0000 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f89fb0700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55a20adf6400 0x55a20ad39700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f89fb0700 1 -- 172.21.15.123:0/2201717452 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55a209fbea80 con 0x55a20add0000 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f96d6b740 4 datascan.main: data pool 'cephfs_data' has ID 6 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f96d6b740 4 datascan.main: opening data pool 'cephfs_data' 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55a20adf6400 0x55a20ad39700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55a20add0400 0x55a20ad37b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:1 7.14 7:287034c4:::1.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add0800 con 0x55a20add0400 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55a20add0400 0x55a20ad37b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.541+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55a20adf6400 0x55a20ad39700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a20acf5da0 tx=0x55a20ad89260 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.542+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55a20add0400 0x55a20ad37b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.542+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== osd_op_reply(1 1.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 160+0+16 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.542+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55a20add0c00 0x55a20ad37600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.542+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:2 7.1f 7:fe07bba1:::100.00000000.inode:head [stat] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1000 con 0x55a20add0c00 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.542+0000 7f2f907bd700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55a20add0c00 0x55a20ad37600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.543+0000 7f2f907bd700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55a20add0c00 0x55a20ad37600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.543+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== osd_op_reply(2 100.00000000.inode [stat out=16b] v0'0 uv1 ondisk = 0) v8 ==== 162+0+16 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.543+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55a20add1400 0x55a20aef8000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.543+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:3 6.0 6:00000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1800 con 0x55a20add1400 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.543+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55a20add1400 0x55a20aef8000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.543+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55a20add1400 0x55a20aef8000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(3 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8afb2700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55a20adf6800 0x55a20aefa680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:4 6.20 6:04000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf6c00 con 0x55a20adf6800 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55a20adf6800 0x55a20aefa680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55a20adf6800 0x55a20aefa680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== osd_op_reply(4 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:5 6.10 6:08000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20add1400 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.544+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== osd_op_reply(5 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.545+0000 7f2f8afb2700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55a20adf7000 0x55a20aefac00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.545+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:6 6.30 6:0c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20adf7000 2023-10-06T04:32:47.607 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.545+0000 7f2f907bd700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55a20adf7000 0x55a20aefac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.545+0000 7f2f907bd700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55a20adf7000 0x55a20aefac00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.545+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 1 ==== osd_op_reply(6 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.545+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:7 6.8 6:10000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7c00 con 0x55a20adf7000 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 2 ==== osd_op_reply(7 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f8afb2700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55a20adf7800 0x55a20aefb180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:8 6.28 6:14000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1800 con 0x55a20adf7800 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55a20adf7800 0x55a20aefb180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f8ffbc700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55a20adf7800 0x55a20aefb180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 1 ==== osd_op_reply(8 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:9 6.18 6:18000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20add1400 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.546+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 3 ==== osd_op_reply(9 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.547+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:10 6.38 6:1c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20add0400 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.547+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== osd_op_reply(10 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.547+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:11 6.4 6:20000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7c00 con 0x55a20adf6800 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.547+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 2 ==== osd_op_reply(11 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.547+0000 7f2f8afb2700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55a20af10400 0x55a20aefb700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.547+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:12 6.24 6:24000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10800 con 0x55a20af10400 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.547+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55a20af10400 0x55a20aefb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.548+0000 7f2f8f7bb700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55a20af10400 0x55a20aefb700 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.548+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 1 ==== osd_op_reply(12 [pgnls-filter start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.548+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:13 6.14 6:28000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1000 con 0x55a20adf7800 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.548+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 2 ==== osd_op_reply(13 [pgnls-filter start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.548+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:14 6.34 6:2c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10c00 con 0x55a20add1400 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.548+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 4 ==== osd_op_reply(14 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.548+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:15 6.c 6:30000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20add0400 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 3 ==== osd_op_reply(15 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:16 6.2c 6:34000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20add0c00 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 2 ==== osd_op_reply(16 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:17 6.1c 6:38000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1800 con 0x55a20adf7000 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 3 ==== osd_op_reply(17 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:18 6.3c 6:3c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11000 con 0x55a20adf7800 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 3 ==== osd_op_reply(18 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f8b7b3700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55a20add1000 0x55a20af1a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.549+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:19 6.2 6:40000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add0800 con 0x55a20add1000 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.550+0000 7f2f907bd700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55a20add1000 0x55a20af1a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.550+0000 7f2f907bd700 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55a20add1000 0x55a20af1a000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.550+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 1 ==== osd_op_reply(19 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.550+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:20 6.22 6:44000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11400 con 0x55a20add0c00 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.550+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 3 ==== osd_op_reply(20 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.550+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:21 6.12 6:48000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20adf7000 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.551+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 4 ==== osd_op_reply(21 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.551+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:22 6.32 6:4c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11800 con 0x55a20add1000 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.551+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 2 ==== osd_op_reply(22 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.551+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:23 6.a 6:50000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11400 con 0x55a20af10400 2023-10-06T04:32:47.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.551+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 2 ==== osd_op_reply(23 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.551+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:24 6.2a 6:54000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf6c00 con 0x55a20add1000 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 3 ==== osd_op_reply(24 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:25 6.1a 6:58000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11c00 con 0x55a20add1000 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 4 ==== osd_op_reply(25 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:26 6.3a 6:5c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10800 con 0x55a20adf6800 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 3 ==== osd_op_reply(26 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:27 6.6 6:60000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1800 con 0x55a20add0400 2023-10-06T04:32:47.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 4 ==== osd_op_reply(27 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.552+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:28 6.26 6:64000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11400 con 0x55a20adf6800 2023-10-06T04:32:47.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.553+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 4 ==== osd_op_reply(28 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.553+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:29 6.16 6:68000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add0800 con 0x55a20add1400 2023-10-06T04:32:47.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.553+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 5 ==== osd_op_reply(29 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.553+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:30 6.36 6:6c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7c00 con 0x55a20af10400 2023-10-06T04:32:47.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.553+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 3 ==== osd_op_reply(30 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.613 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.554+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:31 6.e 6:70000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22000 con 0x55a20add1000 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.554+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 5 ==== osd_op_reply(31 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.554+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:32 6.2e 6:74000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20add0c00 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.554+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 4 ==== osd_op_reply(32 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.554+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:33 6.1e 6:78000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22400 con 0x55a20add1000 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.554+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 6 ==== osd_op_reply(33 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.554+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:34 6.3e 6:7c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22000 con 0x55a20add0400 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.555+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 5 ==== osd_op_reply(34 [pgnls-filter start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.555+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:35 6.1 6:80000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22800 con 0x55a20adf6800 2023-10-06T04:32:47.614 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.555+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 5 ==== osd_op_reply(35 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.555+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:36 6.21 6:84000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23800 con 0x55a20adf7000 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.555+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 5 ==== osd_op_reply(36 [pgnls-filter start_epoch 42 out=49b] v34'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.555+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:37 6.11 6:88000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22c00 con 0x55a20add0400 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.555+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 6 ==== osd_op_reply(37 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.556+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:38 6.31 6:8c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23c00 con 0x55a20af10400 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.556+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 4 ==== osd_op_reply(38 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.556+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:39 6.9 6:90000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23000 con 0x55a20add1400 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.556+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 6 ==== osd_op_reply(39 [pgnls-filter start_epoch 42 out=49b] v34'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.556+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:40 6.29 6:94000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20adf7000 2023-10-06T04:32:47.615 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.556+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 6 ==== osd_op_reply(40 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.556+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:41 6.19 6:98000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23400 con 0x55a20af10400 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.557+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 5 ==== osd_op_reply(41 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.557+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:42 6.39 6:9c000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11800 con 0x55a20af10400 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.557+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 6 ==== osd_op_reply(42 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.557+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:43 6.5 6:a0000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7c00 con 0x55a20adf7000 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.557+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 7 ==== osd_op_reply(43 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.557+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:44 6.25 6:a4000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11c00 con 0x55a20add0c00 2023-10-06T04:32:47.616 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 5 ==== osd_op_reply(44 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:45 6.15 6:a8000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10c00 con 0x55a20adf7000 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 8 ==== osd_op_reply(45 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:46 6.35 6:ac000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf6c00 con 0x55a20af10400 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 7 ==== osd_op_reply(46 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:47 6.d 6:b0000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11000 con 0x55a20af10400 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 8 ==== osd_op_reply(47 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.558+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:48 6.2d 6:b4000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11800 con 0x55a20adf7800 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.559+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 4 ==== osd_op_reply(48 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.559+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:49 6.1d 6:b8000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10800 con 0x55a20adf6800 2023-10-06T04:32:47.617 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.559+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 6 ==== osd_op_reply(49 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.559+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:50 6.3d 6:bc000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20add1400 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.559+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 7 ==== osd_op_reply(50 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.559+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:51 6.3 6:c0000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11400 con 0x55a20adf7000 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.560+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 9 ==== osd_op_reply(51 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.560+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:52 6.23 6:c4000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11c00 con 0x55a20add0c00 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.560+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 6 ==== osd_op_reply(52 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.560+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:53 6.13 6:c8000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add0800 con 0x55a20add0400 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.560+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 7 ==== osd_op_reply(53 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.560+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:54 6.33 6:cc000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf6c00 con 0x55a20af10400 2023-10-06T04:32:47.618 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.561+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 9 ==== osd_op_reply(54 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.561+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:55 6.b 6:d0000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1800 con 0x55a20add0400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.561+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 8 ==== osd_op_reply(55 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.561+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:56 6.2b 6:d4000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20af10400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.561+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 10 ==== osd_op_reply(56 [pgnls-filter start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.561+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:57 6.1b 6:d8000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22000 con 0x55a20add0400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.562+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 9 ==== osd_op_reply(57 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.562+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:58 6.3b 6:dc000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7c00 con 0x55a20adf7800 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.562+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 5 ==== osd_op_reply(58 [pgnls-filter start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.562+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:59 6.7 6:e0000000::::head [pgnls-filter start_epoch 42 in=89b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22400 con 0x55a20af10400 2023-10-06T04:32:47.619 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.562+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 11 ==== osd_op_reply(59 [pgnls-filter start_epoch 42 out=81b] v42'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.562+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:60 6.0 6:00000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1c00 con 0x55a20add1400 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 8 ==== osd_op_reply(60 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:61 6.20 6:04000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22800 con 0x55a20adf6800 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 7 ==== osd_op_reply(61 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:62 6.10 6:08000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22000 con 0x55a20add1400 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 9 ==== osd_op_reply(62 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:63 6.30 6:0c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22c00 con 0x55a20adf7000 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 10 ==== osd_op_reply(63 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.620 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.563+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:64 6.8 6:10000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf6c00 con 0x55a20adf7000 2023-10-06T04:32:47.621 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 11 ==== osd_op_reply(64 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.621 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:65 6.28 6:14000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23000 con 0x55a20adf7800 2023-10-06T04:32:47.621 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 6 ==== osd_op_reply(65 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:66 6.18 6:18000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20add1400 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 10 ==== osd_op_reply(66 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:67 6.38 6:1c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23400 con 0x55a20add0400 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 10 ==== osd_op_reply(67 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:68 6.4 6:20000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add0800 con 0x55a20adf6800 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.564+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 8 ==== osd_op_reply(68 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.565+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:69 6.24 6:24000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23800 con 0x55a20af10400 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.565+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 12 ==== osd_op_reply(69 [pgnls-filter start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.565+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:70 6.14 6:28000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1800 con 0x55a20adf7800 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.565+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 7 ==== osd_op_reply(70 [pgnls-filter start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.622 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.565+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:71 6.34 6:2c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23c00 con 0x55a20add1400 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.565+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 11 ==== osd_op_reply(71 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.565+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:72 6.c 6:30000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11000 con 0x55a20add0400 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.566+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 11 ==== osd_op_reply(72 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.566+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:73 6.2c 6:34000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22000 con 0x55a20add0c00 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.566+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 7 ==== osd_op_reply(73 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.566+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:74 6.1c 6:38000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10800 con 0x55a20adf7000 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.566+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 12 ==== osd_op_reply(74 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.566+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:75 6.3c 6:3c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1c00 con 0x55a20adf7800 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.566+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 8 ==== osd_op_reply(75 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.567+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:76 6.2 6:40000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10c00 con 0x55a20add1000 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.567+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 7 ==== osd_op_reply(76 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.623 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.567+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:77 6.22 6:44000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7c00 con 0x55a20add0c00 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.567+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 8 ==== osd_op_reply(77 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.567+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:78 6.12 6:48000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11400 con 0x55a20adf7000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.568+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 13 ==== osd_op_reply(78 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.568+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:79 6.32 6:4c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11800 con 0x55a20add1000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.568+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 8 ==== osd_op_reply(79 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.568+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:80 6.a 6:50000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11c00 con 0x55a20af10400 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.568+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 13 ==== osd_op_reply(80 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.568+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:81 6.2a 6:54000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20add1000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.568+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 9 ==== osd_op_reply(81 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:82 6.1a 6:58000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10800 con 0x55a20add1000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 10 ==== osd_op_reply(82 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.624 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:83 6.3a 6:5c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24000 con 0x55a20adf6800 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 9 ==== osd_op_reply(83 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:84 6.6 6:60000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22c00 con 0x55a20add0400 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 12 ==== osd_op_reply(84 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:85 6.26 6:64000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24400 con 0x55a20adf6800 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 10 ==== osd_op_reply(85 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.569+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:86 6.16 6:68000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22000 con 0x55a20add1400 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.570+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 12 ==== osd_op_reply(86 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.570+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:87 6.36 6:6c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24800 con 0x55a20af10400 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.570+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 14 ==== osd_op_reply(87 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.570+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:88 6.e 6:70000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf6c00 con 0x55a20add1000 2023-10-06T04:32:47.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.571+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 11 ==== osd_op_reply(88 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.571+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:89 6.2e 6:74000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24c00 con 0x55a20add0c00 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.571+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 9 ==== osd_op_reply(89 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.571+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:90 6.1e 6:78000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23000 con 0x55a20add1000 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.571+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 12 ==== osd_op_reply(90 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.571+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:91 6.3e 6:7c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af25000 con 0x55a20add0400 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.572+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 13 ==== osd_op_reply(91 [pgnls-filter start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.572+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:92 6.1 6:80000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23c00 con 0x55a20adf6800 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.572+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 11 ==== osd_op_reply(92 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.572+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:93 6.21 6:84000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af25400 con 0x55a20adf7000 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.572+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 14 ==== osd_op_reply(93 [pgnls-filter start_epoch 42 out=49b] v34'12 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.573+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:94 6.11 6:88000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22000 con 0x55a20add0400 2023-10-06T04:32:47.626 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.573+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 14 ==== osd_op_reply(94 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.573+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:95 6.31 6:8c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af25800 con 0x55a20af10400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.573+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 15 ==== osd_op_reply(95 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.573+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:96 6.9 6:90000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22400 con 0x55a20add1400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.573+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 13 ==== osd_op_reply(96 [pgnls-filter start_epoch 42 out=49b] v34'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.573+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:97 6.29 6:94000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af25c00 con 0x55a20adf7000 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.574+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 15 ==== osd_op_reply(97 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.574+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:98 6.19 6:98000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22800 con 0x55a20af10400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.574+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 16 ==== osd_op_reply(98 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.574+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:99 6.39 6:9c000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24c00 con 0x55a20af10400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.574+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 17 ==== osd_op_reply(99 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.574+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:100 6.5 6:a0000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1800 con 0x55a20adf7000 2023-10-06T04:32:47.627 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.575+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 16 ==== osd_op_reply(100 [pgnls-filter start_epoch 42 out=49b] v34'5 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.575+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:101 6.25 6:a4000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24000 con 0x55a20add0c00 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.575+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 10 ==== osd_op_reply(101 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.575+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:102 6.15 6:a8000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add1c00 con 0x55a20adf7000 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.575+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 17 ==== osd_op_reply(102 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.575+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:103 6.35 6:ac000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10c00 con 0x55a20af10400 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 18 ==== osd_op_reply(103 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:104 6.d 6:b0000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20add0800 con 0x55a20af10400 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 19 ==== osd_op_reply(104 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:105 6.2d 6:b4000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11400 con 0x55a20adf7800 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 9 ==== osd_op_reply(105 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:106 6.1d 6:b8000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7c00 con 0x55a20adf6800 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.1 v2:172.21.15.123:6816/119720 12 ==== osd_op_reply(106 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20adf6800 2023-10-06T04:32:47.628 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.576+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:107 6.3d 6:bc000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11800 con 0x55a20add1400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.577+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 14 ==== osd_op_reply(107 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.577+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:108 6.3 6:c0000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf6c00 con 0x55a20adf7000 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.577+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 18 ==== osd_op_reply(108 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.577+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:109 6.23 6:c4000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20add0c00 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.577+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 11 ==== osd_op_reply(109 [pgnls-filter start_epoch 42 out=49b] v34'7 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.578+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:110 6.13 6:c8000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20adf7400 con 0x55a20add0400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.578+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 15 ==== osd_op_reply(110 [pgnls-filter start_epoch 42 out=49b] v34'15 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.578+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:111 6.33 6:cc000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10800 con 0x55a20af10400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.578+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 20 ==== osd_op_reply(111 [pgnls-filter start_epoch 42 out=49b] v34'10 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.578+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:112 6.b 6:d0000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24400 con 0x55a20add0400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.578+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 16 ==== osd_op_reply(112 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.578+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:113 6.2b 6:d4000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11000 con 0x55a20af10400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.579+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 21 ==== osd_op_reply(113 [pgnls-filter start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.579+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:114 6.1b 6:d8000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24800 con 0x55a20add0400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.579+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 17 ==== osd_op_reply(114 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.579+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:115 6.3b 6:dc000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af11c00 con 0x55a20adf7800 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.579+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 10 ==== osd_op_reply(115 [pgnls-filter start_epoch 42 out=49b] v34'4 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.579+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:116 6.7 6:e0000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af25000 con 0x55a20af10400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 22 ==== osd_op_reply(116 [pgnls-filter start_epoch 42 out=81b] v42'10 uv10 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 20 datascan.forall_objects: OSD matched oid 10000000001.00000000 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 10 datascan.operator(): handling object 10000000001.0 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:117 6.7 6:e2e07930:::10000000001.00000000:head [getxattr scan_ceiling in=12b,getxattr scan_max_size in=13b,getxattr scan_max_mtime in=14b,getxattr scan_pool_id in=12b,getxattr parent in=6b,getxattr layout in=6b,getxattr symlink in=7b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3a000 con 0x55a20af10400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.5 v2:172.21.15.177:6800/119947 23 ==== osd_op_reply(117 10000000001.00000000 [getxattr (22) out=22b,getxattr (8) out=8b,getxattr (8) out=8b,getxattr,getxattr (95) out=95b,getxattr (30) out=30b,getxattr] v0'0 uv10 ondisk = 0) v8 ==== 416+0+163 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20af10400 2023-10-06T04:32:47.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 20 datascan.operator(): loaded layout from xattr: pi: 6 os: 4194304 sc: 1 su: 4194304 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 10 datascan.inject_with_backtrace: inode: 0x0x10000000001 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 10 datascan.inject_with_backtrace: backptr: 0x0x10000000000/sixmegs 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 20 datascan.get_frag_of: dirino=0x10000000000 target_dname=sixmegs 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.580+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:118 7.10 7:0cf6ad0f:::10000000000.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3a400 con 0x55a20add0400 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.581+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 18 ==== osd_op_reply(118 10000000000.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.581+0000 7f2f96d6b740 4 datascan.get_frag_of: Unexpected error on '10000000000.00000000': (2) No such file or directory 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.581+0000 7f2f96d6b740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x10000000000/sixmegs, will insert to root 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.581+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:119 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3a800 con 0x55a20add0400 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.581+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 19 ==== osd_op_reply(119 10000000000.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 164+0+0 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.581+0000 7f2f96d6b740 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-06T04:32:47.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.581+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:120 7.10 7:0cf6ad0f:::10000000000.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3ac00 con 0x55a20add0400 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 20 ==== osd_op_reply(120 10000000000.00000000 [create,omap-set-header] v42'1 uv1 ondisk = 0) v8 ==== 206+0+0 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f96d6b740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x10000000000 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:121 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3b000 con 0x55a20add0400 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 21 ==== osd_op_reply(121 10000000000.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv1 ondisk = 0) v8 ==== 164+0+4 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f96d6b740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f96d6b740 20 datascan.read_dentry: sixmegs_head not found in result 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f96d6b740 10 datascan.inject_with_backtrace: Linking inode 0x0x10000000001 at 0x0x10000000000/sixmegs with size=6291456 bytes 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.582+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:122 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=496b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3b400 con 0x55a20add0400 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.583+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 22 ==== osd_op_reply(122 10000000000.00000000 [omap-set-vals] v42'2 uv2 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.583+0000 7f2f96d6b740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.583+0000 7f2f96d6b740 10 datascan.inject_with_backtrace: backptr: 0x0x1/subdir 2023-10-06T04:32:47.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.583+0000 7f2f96d6b740 20 datascan.get_frag_of: dirino=0x1 target_dname=subdir 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.583+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:123 7.1f 7:ff5b34d6:::1.00000000:head [getxattr parent in=6b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3b800 con 0x55a20add0c00 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.584+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 12 ==== osd_op_reply(123 1.00000000 [getxattr] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.584+0000 7f2f96d6b740 4 datascan.get_frag_of: Unexpected error on '1.00000000': (2) No such file or directory 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.584+0000 7f2f96d6b740 20 datascan.inject_with_backtrace: don't know fragment for 0x0x1/subdir, will insert to root 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.584+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:124 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-header] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3bc00 con 0x55a20add0c00 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.584+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 13 ==== osd_op_reply(124 1.00000000 [omap-get-header] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.584+0000 7f2f96d6b740 10 datascan.find_or_create_dirfrag: read_version = 0 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.584+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:125 7.1f 7:ff5b34d6:::1.00000000:head [create,omap-set-header in=274b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af25400 con 0x55a20add0c00 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 14 ==== osd_op_reply(125 1.00000000 [create,omap-set-header] v42'2 uv2 ondisk = 0) v8 ==== 196+0+0 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f96d6b740 10 datascan.find_or_create_dirfrag: Created dirfrag: 0x0x1 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:126 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10000 con 0x55a20add0c00 2023-10-06T04:32:47.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 15 ==== osd_op_reply(126 1.00000000 [omap-get-vals-by-keys out=4b] v0'0 uv2 ondisk = 0) v8 ==== 154+0+4 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f96d6b740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f96d6b740 20 datascan.read_dentry: subdir_head not found in result 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f96d6b740 10 datascan.inject_with_backtrace: Linking ancestor directory of inode 0x0x10000000000 at 0x0x1:subdir 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.585+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:127 7.1f 7:ff5b34d6:::1.00000000:head [omap-set-vals in=495b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10c00 con 0x55a20add0c00 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.587+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 16 ==== osd_op_reply(127 1.00000000 [omap-set-vals] v42'3 uv3 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.587+0000 7f2f96d6b740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.587+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:128 6.27 6:e4000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af10800 con 0x55a20add0c00 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.588+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.6 v2:172.21.15.177:6816/119976 17 ==== osd_op_reply(128 [pgnls-filter start_epoch 42 out=49b] v34'2 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add0c00 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.588+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:129 6.17 6:e8000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af25800 con 0x55a20add1000 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.588+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.4 v2:172.21.15.177:6808/119946 13 ==== osd_op_reply(129 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20add1000 2023-10-06T04:32:47.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.588+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:130 6.37 6:ec000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23400 con 0x55a20add1400 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.588+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.0 v2:172.21.15.123:6808/119718 15 ==== osd_op_reply(130 [pgnls-filter start_epoch 42 out=49b] v34'11 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20add1400 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.588+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:131 6.f 6:f0000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af24c00 con 0x55a20adf7800 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.589+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 <== osd.2 v2:172.21.15.123:6824/119723 11 ==== osd_op_reply(131 [pgnls-filter start_epoch 42 out=49b] v34'9 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57440 con 0x55a20adf7800 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.589+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:132 6.2f 6:f4000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af23800 con 0x55a20adf7000 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.589+0000 7f2f907bd700 1 -- 172.21.15.123:0/2201717452 <== osd.7 v2:172.21.15.177:6824/119977 19 ==== osd_op_reply(132 [pgnls-filter start_epoch 42 out=49b] v34'9 uv1 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad57200 con 0x55a20adf7000 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.589+0000 7f2f8afb2700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:133 6.1f 6:f8000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af22c00 con 0x55a20add0400 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.589+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 23 ==== osd_op_reply(133 [pgnls-filter start_epoch 42 out=49b] v34'6 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.589+0000 7f2f8b7b3700 1 -- 172.21.15.123:0/2201717452 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:134 6.3f 6:fc000000::::head [pgnls-filter start_epoch 42 in=85b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55a20af3b800 con 0x55a20add0400 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f8f7bb700 1 -- 172.21.15.123:0/2201717452 <== osd.3 v2:172.21.15.123:6800/119719 24 ==== osd_op_reply(134 [pgnls-filter start_epoch 42 out=49b] v34'8 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55a20ad56fc0 con 0x55a20add0400 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55a20add1400 msgr2=0x55a20aef8000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55a20add1400 0x55a20aef8000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55a20adf6800 msgr2=0x55a20aefa680 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55a20adf6800 0x55a20aefa680 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55a20adf7800 msgr2=0x55a20aefb180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55a20adf7800 0x55a20aefb180 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55a20add0400 msgr2=0x55a20ad37b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55a20add0400 0x55a20ad37b80 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 reap_dead start 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55a20add1000 msgr2=0x55a20af1a000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55a20add1000 0x55a20af1a000 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55a20af10400 msgr2=0x55a20aefb700 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55a20af10400 0x55a20aefb700 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55a20add0c00 msgr2=0x55a20ad37600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f8ffbc700 1 -- 172.21.15.123:0/2201717452 reap_dead start 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55a20add0c00 0x55a20ad37600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55a20adf7000 msgr2=0x55a20aefac00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55a20adf7000 0x55a20aefac00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55a20adf6400 msgr2=0x55a20ad39700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55a20adf6400 0x55a20ad39700 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x55a20acf5da0 tx=0x55a20ad89260 comp rx=0 tx=0).stop 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20add0000 msgr2=0x55a20ad38100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20add0000 0x55a20ad38100 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x55a20ad598c0 tx=0x55a20ad42180 comp rx=0 tx=0).stop 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 shutdown_connections 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55a20adf7000 0x55a20aefac00 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20add0000 0x55a20ad38100 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55a20adf6400 0x55a20ad39700 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/2201717452 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55a20add0c00 0x55a20ad37600 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 >> 172.21.15.123:0/2201717452 conn(0x55a20a0f1000 msgr2=0x55a20a05b800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 shutdown_connections 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/2201717452 wait complete. 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/869629429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20a0f0c00 msgr2=0x55a20ad37080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20a0f0c00 0x55a20ad37080 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x55a20ad586c0 tx=0x55a20ad0d2c0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/869629429 shutdown_connections 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55a20a0f0c00 0x55a20ad37080 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55a20a0f0400 0x55a20ad36580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 --2- 172.21.15.123:0/869629429 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55a20a0f0800 0x55a20ad36b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.590+0000 7f2f96d6b740 1 -- 172.21.15.123:0/869629429 >> 172.21.15.123:0/869629429 conn(0x55a20a0f0000 msgr2=0x55a20a05b000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.591+0000 7f2f96d6b740 1 -- 172.21.15.123:0/869629429 shutdown_connections 2023-10-06T04:32:47.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.591+0000 7f2f96d6b740 1 -- 172.21.15.123:0/869629429 wait complete. 2023-10-06T04:32:47.639 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_inodes', '--alternate-pool', 'recovery_fs_metadata', '--filesystem', 'cephfs', '--force-corrupt', '--force-init', 'cephfs_data'] in time 0:00:00.168549, result: 2023-10-06T04:32:47.639 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-data-scan --debug-mds=20 --debug-ms=1 --debug-objecter=1 scan_links --filesystem recovery_fs 2023-10-06T04:32:47.678 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.676+0000 7f078d9d7740 1 -- start start 2023-10-06T04:32:47.679 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.676+0000 7f078d9d7740 1 Processor -- start 2023-10-06T04:32:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbc800 0x55f85abfeb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.683 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbcc00 0x55f85abff080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f859fbd000 0x55f85abff600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f078d9d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f859e87cc0 con 0x55f859fbcc00 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f078d9d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f859e87540 con 0x55f859fbd000 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f078d9d7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f859e87e00 con 0x55f859fbc800 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbcc00 0x55f85abff080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbcc00 0x55f85abff080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33918/0 (socket says 172.21.15.123:33918) 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 -- 172.21.15.123:0/3023379113 learned_addr learned my addr 172.21.15.123:0/3023379113 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0787429700 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f859fbd000 0x55f85abff600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786c28700 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbc800 0x55f85abfeb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 -- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbc800 msgr2=0x55f85abfeb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbc800 0x55f85abfeb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.684 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 -- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f859fbd000 msgr2=0x55f85abff600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f859fbd000 0x55f85abff600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.677+0000 7f0786427700 1 -- 172.21.15.123:0/3023379113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f859e6a340 con 0x55f859fbcc00 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f0786427700 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbcc00 0x55f85abff080 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55f85ac1a6f0 tx=0x55f85ac26120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31c87613f85f1425 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f0784423700 1 -- 172.21.15.123:0/3023379113 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f859e7f600 con 0x55f859fbcc00 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f0784423700 1 -- 172.21.15.123:0/3023379113 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f859e9c600 con 0x55f859fbcc00 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f0784423700 1 -- 172.21.15.123:0/3023379113 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f859e7f1e0 con 0x55f859fbcc00 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f078d9d7740 1 -- 172.21.15.123:0/3023379113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f859e6a000 con 0x55f859fbcc00 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f078d9d7740 1 -- 172.21.15.123:0/3023379113 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f859e6aea0 con 0x55f859fbcc00 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f078d9d7740 4 waiting for MDS map... 2023-10-06T04:32:47.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f0784423700 1 -- 172.21.15.123:0/3023379113 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55f859e8ac40 con 0x55f859fbcc00 2023-10-06T04:32:47.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f0784423700 1 -- 172.21.15.123:0/3023379113 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x55f85ac3e600 con 0x55f859fbcc00 2023-10-06T04:32:47.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f078d9d7740 4 Got MDS map 22 2023-10-06T04:32:47.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f078d9d7740 4 datascan.main: Using metadata pool output 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.678+0000 7f078d9d7740 4 datascan.main: connecting to RADOS... 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 Processor -- start 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 -- start start 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbdc00 0x55f85abffb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f85ac98000 0x55f85ac00100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85ac98400 0x55f85ac00680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f859e87b80 con 0x55f85ac98000 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f859e87540 con 0x55f85ac98400 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f078d9d7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f859e86dc0 con 0x55f859fbdc00 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.679+0000 7f0786c28700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f85ac98000 0x55f85ac00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0787429700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbdc00 0x55f85abffb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f85ac98000 0x55f85ac00100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33920/0 (socket says 172.21.15.123:33920) 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 -- 172.21.15.123:0/1325276844 learned_addr learned my addr 172.21.15.123:0/1325276844 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786427700 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85ac98400 0x55f85ac00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 -- 172.21.15.123:0/1325276844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbdc00 msgr2=0x55f85abffb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbdc00 0x55f85abffb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 -- 172.21.15.123:0/1325276844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85ac98400 msgr2=0x55f85ac00680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85ac98400 0x55f85ac00680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 -- 172.21.15.123:0/1325276844 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f859e6b860 con 0x55f85ac98000 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f0786c28700 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f85ac98000 0x55f85ac00100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55f85ac1b6e0 tx=0x55f85abd9a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7332dac852a995de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f078241f700 1 -- 172.21.15.123:0/1325276844 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f859e7f080 con 0x55f85ac98000 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f078241f700 1 -- 172.21.15.123:0/1325276844 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f859e9c300 con 0x55f85ac98000 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f078241f700 1 -- 172.21.15.123:0/1325276844 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f859e7f600 con 0x55f85ac98000 2023-10-06T04:32:47.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f078d9d7740 1 -- 172.21.15.123:0/1325276844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f85ac98000 msgr2=0x55f85ac00100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f078d9d7740 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f85ac98000 0x55f85ac00100 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x55f85ac1b6e0 tx=0x55f85abd9a40 comp rx=0 tx=0).stop 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f078d9d7740 1 -- 172.21.15.123:0/1325276844 shutdown_connections 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.680+0000 7f078d9d7740 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85ac98400 0x55f85ac00680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.681+0000 7f078d9d7740 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbdc00 0x55f85abffb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.681+0000 7f078d9d7740 1 --2- 172.21.15.123:0/1325276844 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f85ac98000 0x55f85ac00100 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.681+0000 7f078d9d7740 1 -- 172.21.15.123:0/1325276844 >> 172.21.15.123:0/1325276844 conn(0x55f859fbd800 msgr2=0x55f859f27800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.681+0000 7f078d9d7740 1 -- 172.21.15.123:0/1325276844 shutdown_connections 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.681+0000 7f078d9d7740 1 -- 172.21.15.123:0/1325276844 wait complete. 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.681+0000 7f078d9d7740 1 Processor -- start 2023-10-06T04:32:47.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.681+0000 7f078d9d7740 1 -- start start 2023-10-06T04:32:47.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbd800 0x55f85ac00100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85acb6000 0x55f85ac00680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f078d9d7740 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f85acb6400 0x55f85ac00c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f078d9d7740 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f859e863c0 con 0x55f859fbd800 2023-10-06T04:32:47.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f078d9d7740 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f859e86000 con 0x55f85acb6000 2023-10-06T04:32:47.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f078d9d7740 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f859e87180 con 0x55f85acb6400 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0787429700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbd800 0x55f85ac00100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0787429700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbd800 0x55f85ac00100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33934/0 (socket says 172.21.15.123:33934) 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 learned_addr learned my addr 172.21.15.123:0/374961083 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f85acb6400 0x55f85ac00c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85acb6000 0x55f85ac00680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f85acb6400 msgr2=0x55f85ac00c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f85acb6400 0x55f85ac00c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85acb6000 msgr2=0x55f85ac00680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.682+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f85acb6000 0x55f85ac00680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f859e6a9c0 con 0x55f859fbd800 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbd800 0x55f85ac00100 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55f85ac986c0 tx=0x55f85ac4cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f0780c1c700 1 -- 172.21.15.123:0/374961083 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f859e7f080 con 0x55f859fbd800 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f0780c1c700 1 -- 172.21.15.123:0/374961083 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f859e9cd80 con 0x55f859fbd800 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f859e6ba00 con 0x55f859fbd800 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f0780c1c700 1 -- 172.21.15.123:0/374961083 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f859e7f600 con 0x55f859fbd800 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f859e6a9c0 con 0x55f859fbd800 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.683+0000 7f078d9d7740 4 datascan.init: resolving metadata pool 7 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f0780c1c700 1 -- 172.21.15.123:0/374961083 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55f85abbd080 con 0x55f859fbd800 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f0780c1c700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f85acba400 0x55f85ac01700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f0780c1c700 1 -- 172.21.15.123:0/374961083 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55f859e8aa80 con 0x55f859fbd800 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f078d9d7740 4 datascan.init: found metadata pool 'recovery_fs_metadata' 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f078d9d7740 4 datascan.main: resolving metadata pool 7 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f85acba400 0x55f85ac01700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55f85acb6800 0x55f85ac01180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.704 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.684+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:1 7.0 7:00000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb6c00 con 0x55f85acb6800 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.685+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55f85acb6800 0x55f85ac01180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.685+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f85acba400 0x55f85ac01700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55f85ac1b6e0 tx=0x55f85abd99e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.685+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55f85acb6800 0x55f85ac01180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.685+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.5 v2:172.21.15.177:6800/119947 1 ==== osd_op_reply(1 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acb6800 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.685+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f85acb6c00 0x55f85adac000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.685+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:2 7.10 7:08000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acba800 con 0x55f85acb6c00 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.685+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f85acb6c00 0x55f85adac000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.686+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f85acb6c00 0x55f85adac000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.686+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== osd_op_reply(2 [pgnls start_epoch 42 out=81b] v42'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.686+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55f85acba800 0x55f85adac580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.686+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:3 7.8 7:10000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb000 con 0x55f85acba800 2023-10-06T04:32:47.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.686+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55f85acba800 0x55f85adac580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.686+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55f85acba800 0x55f85adac580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.2 v2:172.21.15.123:6824/119723 1 ==== osd_op_reply(3 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acba800 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f85acbb000 0x55f85abfe000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:4 7.18 7:18000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f859fbc400 con 0x55f85acbb000 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f85acbb000 0x55f85abfe000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f85acbb000 0x55f85abfe000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(4 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:5 7.4 7:20000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f859fbc400 con 0x55f85acbb000 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== osd_op_reply(5 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.687+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:6 7.14 7:28000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f859fbc400 con 0x55f85acb6c00 2023-10-06T04:32:47.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.688+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== osd_op_reply(6 [pgnls start_epoch 42 out=77b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.688+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55f859fbc400 0x55f85adacb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.688+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:7 7.c 7:30000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb400 con 0x55f859fbc400 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.688+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55f859fbc400 0x55f85adacb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.688+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55f859fbc400 0x55f85adacb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== osd_op_reply(7 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:8 7.1c 7:38000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb400 con 0x55f85acb6800 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.5 v2:172.21.15.177:6800/119947 2 ==== osd_op_reply(8 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acb6800 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:9 7.2 7:40000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb400 con 0x55f85acb6c00 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 3 ==== osd_op_reply(9 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:10 7.12 7:48000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb400 con 0x55f859fbc400 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 2 ==== osd_op_reply(10 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.689+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:11 7.a 7:50000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb400 con 0x55f85acb6c00 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.690+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 4 ==== osd_op_reply(11 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.690+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55f85acbb400 0x55f85adad080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.690+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:12 7.1a 7:58000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb800 con 0x55f85acbb400 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.690+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55f85acbb400 0x55f85adad080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.690+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55f85acbb400 0x55f85adad080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.690+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.4 v2:172.21.15.177:6808/119946 1 ==== osd_op_reply(12 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acbb400 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.690+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:13 7.6 7:60000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb800 con 0x55f85acbb000 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 3 ==== osd_op_reply(13 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:14 7.16 7:68000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb800 con 0x55f85acbb400 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.4 v2:172.21.15.177:6808/119946 2 ==== osd_op_reply(14 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acbb400 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:15 7.e 7:70000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acbb800 con 0x55f85acba800 2023-10-06T04:32:47.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.2 v2:172.21.15.123:6824/119723 2 ==== osd_op_reply(15 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acba800 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55f85acbb800 0x55f85adad600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:16 7.1e 7:78000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0000 con 0x55f85acbb800 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.691+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55f85acbb800 0x55f85adad600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.692+0000 7f0786427700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55f85acbb800 0x55f85adad600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.692+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== osd_op_reply(16 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb800 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.692+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:17 7.1 7:80000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0000 con 0x55f85acbb400 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.692+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.4 v2:172.21.15.177:6808/119946 3 ==== osd_op_reply(17 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acbb400 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.692+0000 7f0786c28700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55f85adc0000 0x55f85adadb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.692+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:18 7.11 7:88000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.692+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55f85adc0000 0x55f85adadb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.693+0000 7f0787429700 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55f85adc0000 0x55f85adadb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.693+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 1 ==== osd_op_reply(18 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.693+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:19 7.9 7:90000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.693+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 4 ==== osd_op_reply(19 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.693+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:20 7.19 7:98000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 3 ==== osd_op_reply(20 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:21 7.5 7:a0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 2 ==== osd_op_reply(21 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:22 7.15 7:a8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 3 ==== osd_op_reply(22 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:23 7.d 7:b0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 4 ==== osd_op_reply(23 [pgnls start_epoch 42 out=74b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.710 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.694+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:24 7.1d 7:b8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb800 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.1 v2:172.21.15.123:6816/119720 2 ==== osd_op_reply(24 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb800 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:25 7.3 7:c0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acb6c00 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 5 ==== osd_op_reply(25 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:26 7.13 7:c8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 5 ==== osd_op_reply(26 [pgnls start_epoch 42 out=98b] v42'2 uv2 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:27 7.b 7:d0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb800 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.1 v2:172.21.15.123:6816/119720 3 ==== osd_op_reply(27 [pgnls start_epoch 42 out=76b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb800 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.695+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:28 7.1b 7:d8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 6 ==== osd_op_reply(28 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:29 7.7 7:e0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 5 ==== osd_op_reply(29 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:30 7.17 7:e8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.711 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 4 ==== osd_op_reply(30 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:31 7.f 7:f0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 7 ==== osd_op_reply(31 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.696+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:32 7.1f 7:f8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 6 ==== osd_op_reply(32 [pgnls start_epoch 42 out=101b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object 10000000000.00000000 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:33 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb7000 con 0x55f85acb6c00 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 6 ==== osd_op_reply(33 10000000000.00000000 [omap-get-vals out=497b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+497 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 20 datascan.scan_links: injected ino detected 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object 1.00000000.inode 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object mds_snaptable 2023-10-06T04:32:47.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object mds0_inotable 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object 100.00000000 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:34 7.13 7:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb7400 con 0x55f85acbb000 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 8 ==== osd_op_reply(34 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object mds0_sessionmap 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object 100.00000000.inode 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 10 datascan.scan_links: step 1: handling object 1.00000000 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.697+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:35 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb7800 con 0x55f859fbc400 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 7 ==== osd_op_reply(35 1.00000000 [omap-get-vals out=496b] v0'0 uv3 ondisk = 0) v8 ==== 154+0+496 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.713 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f078d9d7740 20 datascan.scan_links: injected ino detected 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:36 7.0 7:00000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb7c00 con 0x55f85acb6800 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.5 v2:172.21.15.177:6800/119947 3 ==== osd_op_reply(36 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acb6800 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:37 7.10 7:08000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb7c00 con 0x55f85acb6c00 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 7 ==== osd_op_reply(37 [pgnls start_epoch 42 out=81b] v42'2 uv2 ondisk = 1) v8 ==== 144+0+81 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:38 7.8 7:10000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acba800 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.2 v2:172.21.15.123:6824/119723 3 ==== osd_op_reply(38 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acba800 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.698+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:39 7.18 7:18000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 9 ==== osd_op_reply(39 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:40 7.4 7:20000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 10 ==== osd_op_reply(40 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:41 7.14 7:28000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acb6c00 2023-10-06T04:32:47.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 8 ==== osd_op_reply(41 [pgnls start_epoch 42 out=77b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+77 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:42 7.c 7:30000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 8 ==== osd_op_reply(42 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.699+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- osd_op(unknown.0.0:43 7.1c 7:38000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acb6800 2023-10-06T04:32:47.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.5 v2:172.21.15.177:6800/119947 4 ==== osd_op_reply(43 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acb6800 2023-10-06T04:32:47.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:44 7.2 7:40000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acb6c00 2023-10-06T04:32:47.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 9 ==== osd_op_reply(44 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.715 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:45 7.12 7:48000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 9 ==== osd_op_reply(45 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:46 7.a 7:50000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acb6c00 2023-10-06T04:32:47.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 10 ==== osd_op_reply(46 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.700+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:47 7.1a 7:58000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb400 2023-10-06T04:32:47.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.701+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.4 v2:172.21.15.177:6808/119946 4 ==== osd_op_reply(47 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acbb400 2023-10-06T04:32:47.724 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.701+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:48 7.6 7:60000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.701+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 11 ==== osd_op_reply(48 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.701+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:49 7.16 7:68000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb400 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.701+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.4 v2:172.21.15.177:6808/119946 5 ==== osd_op_reply(49 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acbb400 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.701+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- osd_op(unknown.0.0:50 7.e 7:70000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acba800 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.2 v2:172.21.15.123:6824/119723 4 ==== osd_op_reply(50 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acba800 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:51 7.1e 7:78000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb800 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.1 v2:172.21.15.123:6816/119720 4 ==== osd_op_reply(51 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb800 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:52 7.1 7:80000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb400 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 <== osd.4 v2:172.21.15.177:6808/119946 6 ==== osd_op_reply(52 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19440 con 0x55f85acbb400 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:53 7.11 7:88000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.725 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 5 ==== osd_op_reply(53 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.702+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:54 7.9 7:90000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 12 ==== osd_op_reply(54 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:55 7.19 7:98000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 10 ==== osd_op_reply(55 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:56 7.5 7:a0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 6 ==== osd_op_reply(56 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:57 7.15 7:a8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 7 ==== osd_op_reply(57 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.703+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:58 7.d 7:b0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 11 ==== osd_op_reply(58 [pgnls start_epoch 42 out=74b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+74 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.726 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:59 7.1d 7:b8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb800 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.1 v2:172.21.15.123:6816/119720 5 ==== osd_op_reply(59 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb800 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:60 7.3 7:c0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acb6c00 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 11 ==== osd_op_reply(60 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:61 7.13 7:c8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 13 ==== osd_op_reply(61 [pgnls start_epoch 42 out=98b] v42'2 uv2 ondisk = 1) v8 ==== 144+0+98 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.704+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:62 7.b 7:d0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb800 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.1 v2:172.21.15.123:6816/119720 6 ==== osd_op_reply(62 [pgnls start_epoch 42 out=76b] v42'1 uv1 ondisk = 1) v8 ==== 144+0+76 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb800 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:63 7.1b 7:d8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 14 ==== osd_op_reply(63 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:64 7.7 7:e0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.727 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 12 ==== osd_op_reply(64 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- osd_op(unknown.0.0:65 7.17 7:e8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85adc0000 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.7 v2:172.21.15.177:6824/119977 8 ==== osd_op_reply(65 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac19200 con 0x55f85adc0000 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.705+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:66 7.f 7:f0000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acbb000 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 15 ==== osd_op_reply(66 [pgnls start_epoch 42 out=49b] v0'0 uv0 ondisk = 1) v8 ==== 144+0+49 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:67 7.1f 7:f8000000::::head [pgnls start_epoch 42 in=39b] snapc 0=[] ondisk+read+ignore_overlay+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f859fbc400 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 13 ==== osd_op_reply(67 [pgnls start_epoch 42 out=101b] v42'3 uv3 ondisk = 1) v8 ==== 144+0+101 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object 10000000000.00000000 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:68 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0c00 con 0x55f85acb6c00 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 12 ==== osd_op_reply(68 10000000000.00000000 [omap-get-vals out=497b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+497 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.728 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 20 datascan.scan_links: injected ino detected 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 20 datascan.scan_links: adding 0x10000000001 for future processing to fix dnfirst 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object 1.00000000.inode 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: '1.00000000.inode' 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object mds_snaptable 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: 'mds_snaptable' 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object mds0_inotable 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: 'mds0_inotable' 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object 100.00000000 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.706+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:69 7.13 7:cd5a64a3:::100.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc1000 con 0x55f85acbb000 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 16 ==== osd_op_reply(69 100.00000000 [omap-get-vals out=5b] v0'0 uv1 ondisk = 0) v8 ==== 156+0+5 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object mds0_sessionmap 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: 'mds0_sessionmap' 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object 100.00000000.inode 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: Not a dirfrag: '100.00000000.inode' 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: step 2: handling object 1.00000000 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:70 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals in=16b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc1400 con 0x55f859fbc400 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 14 ==== osd_op_reply(70 1.00000000 [omap-get-vals out=496b] v0'0 uv3 ondisk = 0) v8 ==== 154+0+496 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 20 datascan.scan_links: injected ino detected 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 20 datascan.scan_links: adding 0x10000000000 for future processing to fix dnfirst 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: processing 0 dup_primaries, 0 remote_links 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: removing dup dentries from 0 objects 2023-10-06T04:32:47.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: processing 0 bad_nlink_inos 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: processing 2 injected_inos 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 10 datascan.scan_links: handling injected_ino 0x10000000000 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x1:subdir 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:71 7.1f 7:ff5b34d6:::1.00000000:head [omap-get-vals-by-keys in=19b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc1800 con 0x55f859fbc400 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 15 ==== osd_op_reply(71 1.00000000 [omap-get-vals-by-keys out=495b] v0'0 uv3 ondisk = 0) v8 ==== 154+0+495 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 20 datascan.read_dentry: oid=1.00000000 dname=subdir frag=*, r=0 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 20 datascan.scan_links: first is now 2 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.707+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:72 7.1f 7:ff5b34d6:::1.00000000:head [omap-set-vals in=495b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc1c00 con 0x55f859fbc400 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.708+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 16 ==== osd_op_reply(72 1.00000000 [omap-set-vals] v42'4 uv4 ondisk = 0) v8 ==== 154+0+0 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.708+0000 7f078d9d7740 20 datascan.inject_linkage: Injected dentry 0x0x1/subdir pointing to 0x0x10000000000 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.708+0000 7f078d9d7740 10 datascan.scan_links: handling injected_ino 0x10000000001 2023-10-06T04:32:47.731 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.708+0000 7f078d9d7740 20 datascan.scan_links: fixing linkage (dnfirst) of 0x10000000000:sixmegs 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.708+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:73 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-get-vals-by-keys in=20b] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0400 con 0x55f85acb6c00 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.709+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 13 ==== osd_op_reply(73 10000000000.00000000 [omap-get-vals-by-keys out=496b] v0'0 uv2 ondisk = 0) v8 ==== 164+0+496 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.709+0000 7f078d9d7740 20 datascan.read_dentry: oid=10000000000.00000000 dname=sixmegs frag=*, r=0 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.709+0000 7f078d9d7740 20 datascan.scan_links: first is now 2 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.709+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:74 7.10 7:0cf6ad0f:::10000000000.00000000:head [omap-set-vals in=496b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc0c00 con 0x55f85acb6c00 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.710+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.3 v2:172.21.15.123:6800/119719 14 ==== osd_op_reply(74 10000000000.00000000 [omap-set-vals] v42'3 uv3 ondisk = 0) v8 ==== 164+0+0 (crc 0 0 0) 0x55f85ac19200 con 0x55f85acb6c00 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.710+0000 7f078d9d7740 20 datascan.inject_linkage: Injected dentry 0x0x10000000000/sixmegs pointing to 0x0x10000000001 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.710+0000 7f078d9d7740 10 datascan.scan_links: updating inotable 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.710+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:75 7.13 7:c91d4a1d:::mds0_inotable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85adc1400 con 0x55f85acbb000 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.711+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 17 ==== osd_op_reply(75 mds0_inotable [read 0~34 out=34b] v0'0 uv2 ondisk = 0) v8 ==== 157+0+34 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.711+0000 7f078d9d7740 10 mds.0.inotable: skip_inos was [0x10000000000~0x10000000000] 2023-10-06T04:32:47.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.711+0000 7f078d9d7740 10 mds.0.inotable: skip_inos now [0x10000000002~0xfffffffffe] 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.711+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:76 7.13 7:c91d4a1d:::mds0_inotable:head [writefull 0~34 in=34b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb7c00 con 0x55f85acbb000 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.712+0000 7f0786427700 1 -- 172.21.15.123:0/374961083 <== osd.0 v2:172.21.15.123:6808/119718 18 ==== osd_op_reply(76 mds0_inotable [writefull 0~34] v42'3 uv3 ondisk = 0) v8 ==== 157+0+0 (crc 0 0 0) 0x55f85ac18fc0 con 0x55f85acbb000 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.712+0000 7f078d9d7740 10 datascan.scan_links: updating snaptable 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.712+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:77 7.d 7:b50e409b:::mds_snaptable:head [read 0~0] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f85acb7800 con 0x55f859fbc400 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f0787429700 1 -- 172.21.15.123:0/374961083 <== osd.6 v2:172.21.15.177:6816/119976 17 ==== osd_op_reply(77 mds_snaptable [read 0~70 out=70b] v0'0 uv1 ondisk = 0) v8 ==== 157+0+70 (crc 0 0 0) 0x55f85ac19200 con 0x55f859fbc400 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f85acbb000 msgr2=0x55f85abfe000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f85acbb000 0x55f85abfe000 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55f85acbb800 msgr2=0x55f85adad600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x55f85acbb800 0x55f85adad600 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55f85acba800 msgr2=0x55f85adac580 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x55f85acba800 0x55f85adac580 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f85acb6c00 msgr2=0x55f85adac000 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f85acb6c00 0x55f85adac000 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 reap_dead start 2023-10-06T04:32:47.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55f85acbb400 msgr2=0x55f85adad080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x55f85acbb400 0x55f85adad080 crc :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 reap_dead start 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55f85acb6800 msgr2=0x55f85ac01180 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x55f85acb6800 0x55f85ac01180 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55f859fbc400 msgr2=0x55f85adacb00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x55f859fbc400 0x55f85adacb00 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55f85adc0000 msgr2=0x55f85adadb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x55f85adc0000 0x55f85adadb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f85acba400 msgr2=0x55f85ac01700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f85acba400 0x55f85ac01700 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x55f85ac1b6e0 tx=0x55f85abd99e0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbd800 msgr2=0x55f85ac00100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/374961083 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbd800 0x55f85ac00100 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x55f85ac986c0 tx=0x55f85ac4cc00 comp rx=0 tx=0).stop 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 reap_dead start 2023-10-06T04:32:47.735 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f0786c28700 1 -- 172.21.15.123:0/374961083 reap_dead start 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 shutdown_connections 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 >> 172.21.15.123:0/374961083 conn(0x55f859fbdc00 msgr2=0x55f859f27800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 shutdown_connections 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/374961083 wait complete. 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/3023379113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbcc00 msgr2=0x55f85abff080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbcc00 0x55f85abff080 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x55f85ac1a6f0 tx=0x55f85ac26120 comp rx=0 tx=0).stop 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/3023379113 shutdown_connections 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f859fbd000 0x55f85abff600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f859fbc800 0x55f85abfeb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 --2- 172.21.15.123:0/3023379113 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f859fbcc00 0x55f85abff080 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/3023379113 >> 172.21.15.123:0/3023379113 conn(0x55f859fbc000 msgr2=0x55f859f27000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/3023379113 shutdown_connections 2023-10-06T04:32:47.736 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.713+0000 7f078d9d7740 1 -- 172.21.15.123:0/3023379113 wait complete. 2023-10-06T04:32:47.737 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-data-scan', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', 'scan_links', '--filesystem', 'recovery_fs'] in time 0:00:00.097417, result: 2023-10-06T04:32:47.737 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 event recover_dentries list --alternate-pool recovery_fs_metadata 2023-10-06T04:32:47.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.803+0000 7fba27f5c280 1 -- start start 2023-10-06T04:32:47.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.803+0000 7fba27f5c280 1 Processor -- start 2023-10-06T04:32:47.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b62400 0x55f7777a4580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b62800 0x55f7777a4b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b62c00 0x55f7777a5080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba27f5c280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2dcc0 con 0x55f776b62c00 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba27f5c280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2d540 con 0x55f776b62800 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba27f5c280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2de00 con 0x55f776b62400 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b62c00 0x55f7777a5080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b62c00 0x55f7777a5080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33956/0 (socket says 172.21.15.123:33956) 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 -- 172.21.15.123:0/1012191251 learned_addr learned my addr 172.21.15.123:0/1012191251 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba20c0c700 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b62800 0x55f7777a4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 -- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b62400 msgr2=0x55f7777a4580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba2140d700 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b62400 0x55f7777a4580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b62400 0x55f7777a4580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 -- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b62800 msgr2=0x55f7777a4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b62800 0x55f7777a4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 -- 172.21.15.123:0/1012191251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f776a10340 con 0x55f776b62c00 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba20c0c700 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b62800 0x55f7777a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.804+0000 7fba21c0e700 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b62c00 0x55f7777a5080 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55f77776a0c0 tx=0x55f777770fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fe8be9052cc703f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba1ec08700 1 -- 172.21.15.123:0/1012191251 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f776a25600 con 0x55f776b62c00 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba1ec08700 1 -- 172.21.15.123:0/1012191251 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f776a42f00 con 0x55f776b62c00 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba1ec08700 1 -- 172.21.15.123:0/1012191251 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f776a251e0 con 0x55f776b62c00 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba27f5c280 1 -- 172.21.15.123:0/1012191251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f776a10000 con 0x55f776b62c00 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba27f5c280 1 -- 172.21.15.123:0/1012191251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x55f776a10ea0 con 0x55f776b62c00 2023-10-06T04:32:47.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba27f5c280 4 waiting for MDS map... 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba1ec08700 1 -- 172.21.15.123:0/1012191251 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55f776a30c40 con 0x55f776b62c00 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba1ec08700 1 -- 172.21.15.123:0/1012191251 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x55f7777d6600 con 0x55f776b62c00 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba27f5c280 4 Got MDS map 22 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba27f5c280 10 main: JournalTool::main 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba27f5c280 4 main: JournalTool: connecting to RADOS... 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.805+0000 7fba27f5c280 1 Processor -- start 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba27f5c280 1 -- start start 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b63800 0x55f7777a5600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63c00 0x55f7777a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f777842000 0x55f7777a6100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba27f5c280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2db80 con 0x55f776b63800 2023-10-06T04:32:47.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba27f5c280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2d540 con 0x55f776b63c00 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba27f5c280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2cdc0 con 0x55f777842000 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b63800 0x55f7777a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b63800 0x55f7777a5600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33964/0 (socket says 172.21.15.123:33964) 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 -- 172.21.15.123:0/3868083659 learned_addr learned my addr 172.21.15.123:0/3868083659 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba2140d700 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63c00 0x55f7777a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f777842000 0x55f7777a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 -- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f777842000 msgr2=0x55f7777a6100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f777842000 0x55f7777a6100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 -- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63c00 msgr2=0x55f7777a5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63c00 0x55f7777a5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba20c0c700 1 -- 172.21.15.123:0/3868083659 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f776a11860 con 0x55f776b63800 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.806+0000 7fba2140d700 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63c00 0x55f7777a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b63800 0x55f7777a5600 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55f77776b6b0 tx=0x55f77784e0c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea58a08597c6799b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba1cc04700 1 -- 172.21.15.123:0/3868083659 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f776a25600 con 0x55f776b63800 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba1cc04700 1 -- 172.21.15.123:0/3868083659 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f776a42600 con 0x55f776b63800 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba1cc04700 1 -- 172.21.15.123:0/3868083659 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f776a251e0 con 0x55f776b63800 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 -- 172.21.15.123:0/3868083659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b63800 msgr2=0x55f7777a5600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b63800 0x55f7777a5600 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x55f77776b6b0 tx=0x55f77784e0c0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 -- 172.21.15.123:0/3868083659 shutdown_connections 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b63800 0x55f7777a5600 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f777842000 0x55f7777a6100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3868083659 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63c00 0x55f7777a5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 -- 172.21.15.123:0/3868083659 >> 172.21.15.123:0/3868083659 conn(0x55f776b63000 msgr2=0x55f777767800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 -- 172.21.15.123:0/3868083659 shutdown_connections 2023-10-06T04:32:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.807+0000 7fba27f5c280 1 -- 172.21.15.123:0/3868083659 wait complete. 2023-10-06T04:32:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.808+0000 7fba27f5c280 1 Processor -- start 2023-10-06T04:32:47.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.808+0000 7fba27f5c280 1 -- start start 2023-10-06T04:32:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b63c00 0x55f7777a6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63800 0x55f7777a6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f777858000 0x55f7777a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2c3c0 con 0x55f777858000 2023-10-06T04:32:47.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2c000 con 0x55f776b63800 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55f776a2d180 con 0x55f776b63c00 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f777858000 0x55f7777a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f777858000 0x55f7777a6100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33980/0 (socket says 172.21.15.123:33980) 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 -- 172.21.15.123:0/3035718853 learned_addr learned my addr 172.21.15.123:0/3035718853 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b63c00 0x55f7777a6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba2140d700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63800 0x55f7777a6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 -- 172.21.15.123:0/3035718853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b63c00 msgr2=0x55f7777a6c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b63c00 0x55f7777a6c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 -- 172.21.15.123:0/3035718853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63800 msgr2=0x55f7777a6680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b63800 0x55f7777a6680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 -- 172.21.15.123:0/3035718853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55f776a10340 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f777858000 0x55f7777a6100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55f77776b920 tx=0x55f77784e1e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba1b401700 1 -- 172.21.15.123:0/3035718853 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f776a24dc0 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba1b401700 1 -- 172.21.15.123:0/3035718853 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55f776a42000 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x55f776a11a00 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba1b401700 1 -- 172.21.15.123:0/3035718853 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x55f776a25600 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55f776a10000 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.809+0000 7fba27f5c280 4 main: JournalTool: resolving pool 7 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba1b401700 1 -- 172.21.15.123:0/3035718853 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x55f777763080 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba1b401700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f777866400 0x55f7777a7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba1b401700 1 -- 172.21.15.123:0/3035718853 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x55f776a30a80 con 0x55f777858000 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba27f5c280 4 main: JournalTool: creating IoCtx.. 2023-10-06T04:32:47.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba27f5c280 4 main: Executing for rank 0 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba2140d700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f777866400 0x55f7777a7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba27f5c280 1 main_event: Using alternate pool recovery_fs_metadata 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f777858400 0x55f7777a5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:1 7.f 7:f1f69726:::400.00000000:head [read 0~2147483647] snapc 0=[] ondisk+read+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f777858800 con 0x55f777858400 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f777858400 0x55f7777a5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba2140d700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f777866400 0x55f7777a7700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55f776a7bc80 tx=0x55f7777b4300 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.811+0000 7fba20c0c700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f777858400 0x55f7777a5b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.812+0000 7fba20c0c700 1 -- 172.21.15.123:0/3035718853 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(1 400.00000000 [read 0~2147483647] v0'0 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x55f7777c2fc0 con 0x55f777858400 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.812+0000 7fba27f5c280 -1 Pointer 400.00000000 is absent 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.812+0000 7fba27f5c280 4 main_event: consumed 0 inodes 2023-10-06T04:32:47.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.812+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f777858c00 0x55f7777a5600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.812+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:2 7.3 7:c0ce80d4:::4.00000000:head [omap-rm-keys in=4b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x55f777859000 con 0x55f777858c00 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.812+0000 7fba21c0e700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f777858c00 0x55f7777a5600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.812+0000 7fba21c0e700 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f777858c00 0x55f7777a5600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba21c0e700 1 -- 172.21.15.123:0/3035718853 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== osd_op_reply(2 4.00000000 [omap-rm-keys] v42'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 154+0+0 (crc 0 0 0) 0x55f7777c3680 con 0x55f777858c00 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 4 0 events 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f777858400 msgr2=0x55f7777a5b80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x55f777858400 0x55f7777a5b80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f777858c00 msgr2=0x55f7777a5600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x55f777858c00 0x55f7777a5600 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f777866400 msgr2=0x55f7777a7700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x55f777866400 0x55f7777a7700 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x55f776a7bc80 tx=0x55f7777b4300 comp rx=0 tx=0).stop 2023-10-06T04:32:47.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f777858000 msgr2=0x55f7777a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 --2- 172.21.15.123:0/3035718853 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f777858000 0x55f7777a6100 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x55f77776b920 tx=0x55f77784e1e0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba2140d700 1 -- 172.21.15.123:0/3035718853 reap_dead start 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 shutdown_connections 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 >> 172.21.15.123:0/3035718853 conn(0x55f776b63000 msgr2=0x55f777767800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba2140d700 1 -- 172.21.15.123:0/3035718853 reap_dead start 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 shutdown_connections 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.813+0000 7fba27f5c280 1 -- 172.21.15.123:0/3035718853 wait complete. 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 -- 172.21.15.123:0/1012191251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b62c00 msgr2=0x55f7777a5080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b62c00 0x55f7777a5080 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x55f77776a0c0 tx=0x55f777770fc0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 -- 172.21.15.123:0/1012191251 shutdown_connections 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55f776b62c00 0x55f7777a5080 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55f776b62400 0x55f7777a4580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 --2- 172.21.15.123:0/1012191251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55f776b62800 0x55f7777a4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 -- 172.21.15.123:0/1012191251 >> 172.21.15.123:0/1012191251 conn(0x55f776b62000 msgr2=0x55f777767000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 -- 172.21.15.123:0/1012191251 shutdown_connections 2023-10-06T04:32:47.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.814+0000 7fba27f5c280 1 -- 172.21.15.123:0/1012191251 wait complete. 2023-10-06T04:32:47.827 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'event', 'recover_dentries', 'list', '--alternate-pool', 'recovery_fs_metadata'] in time 0:00:00.090071, result: 2023-10-06T04:32:47.827 DEBUG:teuthology.orchestra.run.smithi123:> cephfs-journal-tool --debug-mds=20 --debug-ms=1 --debug-objecter=1 --rank recovery_fs:0 journal reset --force 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.898+0000 7f032c356280 1 -- start start 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.898+0000 7f032c356280 1 Processor -- start 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f032c356280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab4400 0x5561a76f6580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab4800 0x5561a76f6b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab4c00 0x5561a76f7080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f032c356280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697fcc0 con 0x5561a6ab4400 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697fe00 con 0x5561a6ab4800 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a697f540 con 0x5561a6ab4c00 2023-10-06T04:32:47.916 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab4400 0x5561a76f6580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab4400 0x5561a76f6580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33982/0 (socket says 172.21.15.123:33982) 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 -- 172.21.15.123:0/4285268561 learned_addr learned my addr 172.21.15.123:0/4285268561 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0326008700 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab4c00 0x5561a76f7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325006700 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab4800 0x5561a76f6b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 -- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab4c00 msgr2=0x5561a76f7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab4c00 0x5561a76f7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 -- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab4800 msgr2=0x5561a76f6b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab4800 0x5561a76f6b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.899+0000 7f0325807700 1 -- 172.21.15.123:0/4285268561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a6962b60 con 0x5561a6ab4400 2023-10-06T04:32:47.917 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f0325807700 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab4400 0x5561a76f6580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5561a76bc0c0 tx=0x5561a69fa120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64b4de682a19319b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f0323002700 1 -- 172.21.15.123:0/4285268561 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a6977600 con 0x5561a6ab4400 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f0323002700 1 -- 172.21.15.123:0/4285268561 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561a6994f00 con 0x5561a6ab4400 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f0323002700 1 -- 172.21.15.123:0/4285268561 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a69771e0 con 0x5561a6ab4400 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f032c356280 1 -- 172.21.15.123:0/4285268561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a6962000 con 0x5561a6ab4400 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f032c356280 1 -- 172.21.15.123:0/4285268561 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5561a6962ea0 con 0x5561a6ab4400 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f032c356280 4 waiting for MDS map... 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f0323002700 1 -- 172.21.15.123:0/4285268561 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x5561a6982c40 con 0x5561a6ab4400 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f0323002700 1 -- 172.21.15.123:0/4285268561 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x5561a770a600 con 0x5561a6ab4400 2023-10-06T04:32:47.918 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f032c356280 4 Got MDS map 22 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f032c356280 10 main: JournalTool::main 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.900+0000 7f032c356280 4 main: JournalTool: connecting to RADOS... 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 Processor -- start 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 -- start start 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab5800 0x5561a76f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5c00 0x5561a76f7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a778c000 0x5561a76f8100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697fb80 con 0x5561a6ab5c00 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697fe00 con 0x5561a778c000 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a697edc0 con 0x5561a6ab5800 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f0325807700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5c00 0x5561a76f7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f0325807700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5c00 0x5561a76f7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33994/0 (socket says 172.21.15.123:33994) 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f0325807700 1 -- 172.21.15.123:0/620442316 learned_addr learned my addr 172.21.15.123:0/620442316 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f0326008700 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab5800 0x5561a76f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.901+0000 7f0325006700 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a778c000 0x5561a76f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0325807700 1 -- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab5800 msgr2=0x5561a76f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0325807700 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab5800 0x5561a76f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0325807700 1 -- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a778c000 msgr2=0x5561a76f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0325807700 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a778c000 0x5561a76f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0325807700 1 -- 172.21.15.123:0/620442316 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a6962b60 con 0x5561a6ab5c00 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0325807700 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5c00 0x5561a76f7b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5561a76bd5f0 tx=0x5561a76c28a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=254069763a96879e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0320ffe700 1 -- 172.21.15.123:0/620442316 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a6976f20 con 0x5561a6ab5c00 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0320ffe700 1 -- 172.21.15.123:0/620442316 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561a773c780 con 0x5561a6ab5c00 2023-10-06T04:32:47.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f0320ffe700 1 -- 172.21.15.123:0/620442316 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a6977600 con 0x5561a6ab5c00 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 -- 172.21.15.123:0/620442316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5c00 msgr2=0x5561a76f7b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5c00 0x5561a76f7b80 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x5561a76bd5f0 tx=0x5561a76c28a0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 -- 172.21.15.123:0/620442316 shutdown_connections 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a778c000 0x5561a76f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab5800 0x5561a76f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 --2- 172.21.15.123:0/620442316 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5c00 0x5561a76f7b80 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 -- 172.21.15.123:0/620442316 >> 172.21.15.123:0/620442316 conn(0x5561a6ab5000 msgr2=0x5561a76b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 -- 172.21.15.123:0/620442316 shutdown_connections 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.902+0000 7f032c356280 1 -- 172.21.15.123:0/620442316 wait complete. 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.903+0000 7f032c356280 1 Processor -- start 2023-10-06T04:32:47.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.903+0000 7f032c356280 1 -- start start 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.903+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab5c00 0x5561a76f8c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5800 0x5561a76f8680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8000 0x5561a76f8100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697e3c0 con 0x5561a6ab5800 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697e000 con 0x5561a6ab5c00 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a697f180 con 0x5561a77a8000 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5800 0x5561a76f8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5800 0x5561a76f8680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34000/0 (socket says 172.21.15.123:34000) 2023-10-06T04:32:47.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0326008700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab5c00 0x5561a76f8c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 -- 172.21.15.123:0/3234406970 learned_addr learned my addr 172.21.15.123:0/3234406970 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325006700 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8000 0x5561a76f8100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 -- 172.21.15.123:0/3234406970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8000 msgr2=0x5561a76f8100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8000 0x5561a76f8100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 -- 172.21.15.123:0/3234406970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab5c00 msgr2=0x5561a76f8c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab5c00 0x5561a76f8c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 -- 172.21.15.123:0/3234406970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a6963860 con 0x5561a6ab5800 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f0325807700 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5800 0x5561a76f8680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5561a76bd860 tx=0x5561a7744cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f031f7fb700 1 -- 172.21.15.123:0/3234406970 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a69771e0 con 0x5561a6ab5800 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f031f7fb700 1 -- 172.21.15.123:0/3234406970 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561a6994000 con 0x5561a6ab5800 2023-10-06T04:32:47.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f031f7fb700 1 -- 172.21.15.123:0/3234406970 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a6976dc0 con 0x5561a6ab5800 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x5561a6962000 con 0x5561a6ab5800 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a6962ea0 con 0x5561a6ab5800 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.904+0000 7f032c356280 4 main: JournalTool: resolving pool 7 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.905+0000 7f031f7fb700 1 -- 172.21.15.123:0/3234406970 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x5561a76b5080 con 0x5561a6ab5800 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f031f7fb700 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5561a77b6800 0x5561a76f9700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f031f7fb700 1 -- 172.21.15.123:0/3234406970 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x5561a6982a80 con 0x5561a6ab5800 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f0326008700 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5561a77b6800 0x5561a76f9700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f032c356280 4 main: JournalTool: creating IoCtx.. 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f032c356280 4 main: Executing for rank 0 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f0326008700 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5561a77b6800 0x5561a76f9700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5561a69cdc80 tx=0x5561a77082a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f032c356280 1 -- start start 2023-10-06T04:32:47.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.906+0000 7f032c356280 1 Processor -- start 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a77a8800 0x5561a76f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f032c356280 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8c00 0x5561a78ea000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f032c356280 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a77a9000 0x5561a78ea580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f032c356280 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697f180 con 0x5561a77a9000 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x5561a697e000 con 0x5561a77a8800 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f032c356280 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x5561a697e3c0 con 0x5561a77a8c00 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f0325006700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a77a9000 0x5561a78ea580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f0325006700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a77a9000 0x5561a78ea580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34008/0 (socket says 172.21.15.123:34008) 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f0325006700 1 -- 172.21.15.123:0/3456261774 learned_addr learned my addr 172.21.15.123:0/3456261774 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f0326008700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a77a8800 0x5561a76f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.907+0000 7f0325807700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8c00 0x5561a78ea000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0325006700 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8c00 msgr2=0x5561a78ea000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0325006700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8c00 0x5561a78ea000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0325006700 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a77a8800 msgr2=0x5561a76f7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0325006700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a77a8800 0x5561a76f7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0325006700 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5561a77ed380 con 0x5561a77a9000 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0325807700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8c00 0x5561a78ea000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0326008700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a77a8800 0x5561a76f7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0325006700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a77a9000 0x5561a78ea580 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5561a78f20f0 tx=0x5561a78a6c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b94a1c847c22477 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f031c7f5700 1 -- 172.21.15.123:0/3456261774 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a6977600 con 0x5561a77a9000 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f031c7f5700 1 -- 172.21.15.123:0/3456261774 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5561a6994600 con 0x5561a77a9000 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x5561a77ed1e0 con 0x5561a77a9000 2023-10-06T04:32:47.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({fsmap=0}) v3 -- 0x5561a77ed040 con 0x5561a77a9000 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f032c356280 4 waiting for MDS map... 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f031c7f5700 1 -- 172.21.15.123:0/3456261774 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x5561a6976420 con 0x5561a77a9000 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f031c7f5700 1 -- 172.21.15.123:0/3456261774 <== mon.0 v2:172.21.15.123:3300/0 4 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x5561a6982700 con 0x5561a77a9000 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f031c7f5700 1 -- 172.21.15.123:0/3456261774 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 22) v1 ==== 3663+0+0 (secure 0 0 0) 0x5561a78f8600 con 0x5561a77a9000 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f032c356280 4 Got MDS map 22 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5561a77a9400 0x5561a78eab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:1 7.f 7:f1f69726:::400.00000000:head [writefull 0~22 in=22b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a77a9800 con 0x5561a77a9400 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.908+0000 7f0326008700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5561a77a9400 0x5561a78eab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.909+0000 7f0326008700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5561a77a9400 0x5561a78eab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=b1826c1c0d08575d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.910+0000 7f0326008700 1 -- 172.21.15.123:0/3456261774 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== osd_op_reply(1 400.00000000 [writefull 0~22] v42'1 uv1 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f200 con 0x5561a77a9400 2023-10-06T04:32:47.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.910+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x5561a77a9c00 0x5561a78eb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.910+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:2 7.14 7:292cf221:::200.00000000:head [writefull 0~90 in=90b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a77b7000 con 0x5561a77a9c00 2023-10-06T04:32:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.910+0000 7f0325807700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x5561a77a9c00 0x5561a78eb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.910+0000 7f0325807700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x5561a77a9c00 0x5561a78eb080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=7913d0b72389fffc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.911+0000 7f0325807700 1 -- 172.21.15.123:0/3456261774 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== osd_op_reply(2 200.00000000 [writefull 0~90] v42'2 uv2 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f440 con 0x5561a77a9c00 2023-10-06T04:32:47.928 INFO:teuthology.orchestra.run.smithi123.stdout:writing EResetJournal entry 2023-10-06T04:32:47.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:3 7.14 7:2e2fa760:::200.00000001:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a77b7400 con 0x5561a77a9c00 2023-10-06T04:32:47.933 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- osd_op(unknown.0.0:4 7.1b 7:ddb4e4d7:::200.00000002:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a77b7800 con 0x5561a77a9400 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x5561a77b7c00 0x5561a78eb600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:5 7.b 7:d5c7a900:::200.00000003:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a791e000 con 0x5561a77b7c00 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5561a791e400 0x5561a78ebb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- osd_op(unknown.0.0:6 7.1f 7:f83d65d8:::200.00000004:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a791e800 con 0x5561a791e400 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x5561a791ec00 0x5561a78ec100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- osd_op(unknown.0.0:7 7.1a 7:5e41ab69:::200.00000005:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a791f000 con 0x5561a791ec00 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- osd_op(unknown.0.0:8 7.1e 7:7bf7e7b5:::200.00000006:head [delete] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a791f400 con 0x5561a77b7c00 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f0326008700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x5561a77b7c00 0x5561a78eb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f0325807700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5561a791e400 0x5561a78ebb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f0325006700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x5561a791ec00 0x5561a78ec100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f0326008700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x5561a77b7c00 0x5561a78eb600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=a933b05e5c341191 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.912+0000 7f0325807700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5561a791e400 0x5561a78ebb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=827b80066a38651e server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.913+0000 7f0325006700 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x5561a791ec00 0x5561a78ec100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=32f57cc259d09e3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.913+0000 7f0326008700 1 -- 172.21.15.123:0/3456261774 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== osd_op_reply(4 200.00000002 [delete] v42'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f200 con 0x5561a77a9400 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.913+0000 7f0326008700 1 -- 172.21.15.123:0/3456261774 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== osd_op_reply(5 200.00000003 [delete] v42'2 uv1 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f200 con 0x5561a77b7c00 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.914+0000 7f0325807700 1 -- 172.21.15.123:0/3456261774 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== osd_op_reply(6 200.00000004 [delete] v42'5 uv4 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f440 con 0x5561a791e400 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.914+0000 7f0325006700 1 -- 172.21.15.123:0/3456261774 <== osd.4 v2:172.21.15.177:6808/119946 1 ==== osd_op_reply(7 200.00000005 [delete] v42'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a7930000 con 0x5561a791ec00 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.915+0000 7f0325807700 1 -- 172.21.15.123:0/3456261774 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== osd_op_reply(3 200.00000001 [delete] v42'3 uv2 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f440 con 0x5561a77a9c00 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.915+0000 7f031aff2700 1 -- 172.21.15.123:0/3456261774 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- osd_op(unknown.0.0:9 7.14 7:2e2fa760:::200.00000001:head [write 0~48 in=48b] snapc 0=[] ondisk+write+known_if_redirected+supports_pool_eio e42) v8 -- 0x5561a791e800 con 0x5561a77a9c00 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.915+0000 7f0326008700 1 -- 172.21.15.123:0/3456261774 <== osd.1 v2:172.21.15.123:6816/119720 2 ==== osd_op_reply(8 200.00000006 [delete] v42'1 uv0 ondisk = -2 ((2) No such file or directory)) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f200 con 0x5561a77b7c00 2023-10-06T04:32:47.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f0325807700 1 -- 172.21.15.123:0/3456261774 <== osd.3 v2:172.21.15.123:6800/119719 3 ==== osd_op_reply(9 200.00000001 [write 0~48] v42'4 uv4 ondisk = 0) v8 ==== 156+0+0 (crc 0 0 0) 0x5561a771f440 con 0x5561a77a9c00 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 4 Successfully wrote new journal pointer and header for rank 3:0 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5561a77a9400 msgr2=0x5561a78eab00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x5561a77a9400 0x5561a78eab00 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x5561a77b7c00 msgr2=0x5561a78eb600 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x5561a77b7c00 0x5561a78eb600 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x5561a77a9c00 msgr2=0x5561a78eb080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x5561a77a9c00 0x5561a78eb080 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x5561a791ec00 msgr2=0x5561a78ec100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x5561a791ec00 0x5561a78ec100 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f0325006700 1 -- 172.21.15.123:0/3456261774 reap_dead start 2023-10-06T04:32:47.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5561a791e400 msgr2=0x5561a78ebb80 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x5561a791e400 0x5561a78ebb80 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f0325006700 1 -- 172.21.15.123:0/3456261774 reap_dead start 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a77a9000 msgr2=0x5561a78ea580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.916+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a77a9000 0x5561a78ea580 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x5561a78f20f0 tx=0x5561a78a6c00 comp rx=0 tx=0).stop 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 shutdown_connections 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/3456261774 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a77a9000 0x5561a78ea580 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 >> 172.21.15.123:0/3456261774 conn(0x5561a77a8400 msgr2=0x5561a76ba000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 shutdown_connections 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3456261774 wait complete. 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5561a77b6800 msgr2=0x5561a76f9700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5561a77b6800 0x5561a76f9700 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x5561a69cdc80 tx=0x5561a77082a0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5800 msgr2=0x5561a76f8680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5800 0x5561a76f8680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x5561a76bd860 tx=0x5561a7744cc0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 shutdown_connections 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x5561a77b6800 0x5561a76f9700 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a77a8000 0x5561a76f8100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab5c00 0x5561a76f8c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/3234406970 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab5800 0x5561a76f8680 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 >> 172.21.15.123:0/3234406970 conn(0x5561a6ab5000 msgr2=0x5561a76b9800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 shutdown_connections 2023-10-06T04:32:47.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/3234406970 wait complete. 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/4285268561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab4400 msgr2=0x5561a76f6580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab4400 0x5561a76f6580 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x5561a76bc0c0 tx=0x5561a69fa120 comp rx=0 tx=0).stop 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/4285268561 shutdown_connections 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x5561a6ab4400 0x5561a76f6580 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x5561a6ab4c00 0x5561a76f7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 --2- 172.21.15.123:0/4285268561 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x5561a6ab4800 0x5561a76f6b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/4285268561 >> 172.21.15.123:0/4285268561 conn(0x5561a6ab4000 msgr2=0x5561a76b9000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/4285268561 shutdown_connections 2023-10-06T04:32:47.939 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:47.917+0000 7f032c356280 1 -- 172.21.15.123:0/4285268561 wait complete. 2023-10-06T04:32:47.939 DEBUG:tasks.cephfs.filesystem:Ran ['cephfs-journal-tool', '--debug-mds=20', '--debug-ms=1', '--debug-objecter=1', '--rank', 'recovery_fs:0', 'journal', 'reset', '--force'] in time 0:00:00.112334, result: writing EResetJournal entry 2023-10-06T04:32:47.940 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set recovery_fs joinable true 2023-10-06T04:32:48.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.092+0000 7fe8e8f38700 1 Processor -- start 2023-10-06T04:32:48.095 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.092+0000 7fe8e8f38700 1 -- start start 2023-10-06T04:32:48.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e8f38700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fe8e40ae620 0x7fe8e40ca960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:48.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e8f38700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8e40caea0 0x7fe8e40cdef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:48.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e8f38700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8e40cb300 con 0x7fe8e41596d0 2023-10-06T04:32:48.097 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e8f38700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e4159aa0 con 0x7fe8e40caea0 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e8f38700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8e40cf4e0 con 0x7fe8e40ae620 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e2d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8e40caea0 0x7fe8e40cdef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e2d9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8e40caea0 0x7fe8e40cdef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34014/0 (socket says 172.21.15.123:34014) 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.093+0000 7fe8e2d9d700 1 -- 172.21.15.123:0/462177849 learned_addr learned my addr 172.21.15.123:0/462177849 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e1d9b700 1 --2- 172.21.15.123:0/462177849 >> v2:172.21.15.177:3300/0 conn(0x7fe8e40ae620 0x7fe8e40ca960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e2d9d700 1 -- 172.21.15.123:0/462177849 >> v2:172.21.15.177:3300/0 conn(0x7fe8e40ae620 msgr2=0x7fe8e40ca960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e2d9d700 1 --2- 172.21.15.123:0/462177849 >> v2:172.21.15.177:3300/0 conn(0x7fe8e40ae620 0x7fe8e40ca960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e2d9d700 1 -- 172.21.15.123:0/462177849 >> v1:172.21.15.177:6789/0 conn(0x7fe8e41596d0 legacy=0x7fe8e40d0c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e2d9d700 1 -- 172.21.15.123:0/462177849 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8e40cf7c0 con 0x7fe8e40caea0 2023-10-06T04:32:48.098 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e2d9d700 1 --2- 172.21.15.123:0/462177849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e40cdef0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4064230 tx=0x7fe8d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fbfe8a6c1880dad server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e159a700 1 -- 172.21.15.123:0/462177849 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400b820 con 0x7fe8e40caea0 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e159a700 1 -- 172.21.15.123:0/462177849 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8d400b980 con 0x7fe8e40caea0 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.094+0000 7fe8e159a700 1 -- 172.21.15.123:0/462177849 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8d400bd20 con 0x7fe8e40caea0 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 -- 172.21.15.123:0/462177849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 msgr2=0x7fe8e40cdef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/462177849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e40cdef0 secure :-1 s=READY pgs=485 cs=0 l=1 rev1=1 crypto rx=0x7fe8e4064230 tx=0x7fe8d4009250 comp rx=0 tx=0).stop 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 -- 172.21.15.123:0/462177849 shutdown_connections 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/462177849 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e40cdef0 unknown :-1 s=CLOSED pgs=485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/462177849 >> v2:172.21.15.177:3300/0 conn(0x7fe8e40ae620 0x7fe8e40ca960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:48.099 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 -- 172.21.15.123:0/462177849 >> 172.21.15.123:0/462177849 conn(0x7fe8e4063a10 msgr2=0x7fe8e40d03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:48.100 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 -- 172.21.15.123:0/462177849 shutdown_connections 2023-10-06T04:32:48.100 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.095+0000 7fe8e8f38700 1 -- 172.21.15.123:0/462177849 wait complete. 2023-10-06T04:32:48.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.096+0000 7fe8e8f38700 1 Processor -- start 2023-10-06T04:32:48.102 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.096+0000 7fe8e8f38700 1 -- start start 2023-10-06T04:32:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40ae620 0x7fe8e414e1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e416da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e41596d0 0x7fe8e415f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e4169ac0 con 0x7fe8e40caea0 2023-10-06T04:32:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e414e6f0 con 0x7fe8e40ae620 2023-10-06T04:32:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8e40b20e0 con 0x7fe8e41596d0 2023-10-06T04:32:48.103 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e416da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e416da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34026/0 (socket says 172.21.15.123:34026) 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 -- 172.21.15.123:0/1989810908 learned_addr learned my addr 172.21.15.123:0/1989810908 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e259c700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40ae620 0x7fe8e414e1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e2d9d700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e41596d0 0x7fe8e415f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 -- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e41596d0 msgr2=0x7fe8e415f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e41596d0 0x7fe8e415f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 -- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40ae620 msgr2=0x7fe8e414e1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40ae620 0x7fe8e414e1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 -- 172.21.15.123:0/1989810908 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8d40094f0 con 0x7fe8e40caea0 2023-10-06T04:32:48.104 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e1d9b700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e416da80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00d860 tx=0x7fe8cc008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8da7fc700 1 -- 172.21.15.123:0/1989810908 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc0125f0 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8da7fc700 1 -- 172.21.15.123:0/1989810908 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8cc012750 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8da7fc700 1 -- 172.21.15.123:0/1989810908 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8cc012b10 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8e4169cd0 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.097+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8e416a290 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.098+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8e406a230 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.099+0000 7fe8da7fc700 1 -- 172.21.15.123:0/1989810908 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8cc01e460 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.099+0000 7fe8da7fc700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8c4066f30 0x7fe8c40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.099+0000 7fe8da7fc700 1 -- 172.21.15.123:0/1989810908 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fe8cc084de0 con 0x7fe8e40caea0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.099+0000 7fe8e259c700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8c4066f30 0x7fe8c40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:48.105 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.099+0000 7fe8e259c700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8c4066f30 0x7fe8c40693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00f430 tx=0x7fe8dc002ad0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:48.106 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.103+0000 7fe8da7fc700 1 -- 172.21.15.123:0/1989810908 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8cc055760 con 0x7fe8e40caea0 2023-10-06T04:32:48.333 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:48.330+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"} v 0) v1 -- 0x7fe8e40cbcd0 con 0x7fe8e40caea0 2023-10-06T04:32:49.251 INFO:tasks.ceph.mon.a.smithi123.stderr:2023-10-06T04:32:49.249+0000 7fea93165700 -1 log_channel(cluster) log [ERR] : Health check update: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:32:49.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.253+0000 7fe8da7fc700 1 -- 172.21.15.123:0/1989810908 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "recovery_fs", "var": "joinable", "val": "true"}]=0 recovery_fs marked joinable; MDS may join as newly active. v23) v1 ==== 172+0+0 (secure 0 0 0) 0x7fe8cc0552f0 con 0x7fe8e40caea0 2023-10-06T04:32:49.257 INFO:tasks.ceph.mds.a.smithi123.stderr:2023-10-06T04:32:49.255+0000 7f99813f7700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-06T04:32:49.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.256+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8c4066f30 msgr2=0x7fe8c40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:49.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.256+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8c4066f30 0x7fe8c40693e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fe8dc00f430 tx=0x7fe8dc002ad0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 msgr2=0x7fe8e416da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e416da80 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fe8cc00d860 tx=0x7fe8cc008040 comp rx=0 tx=0).stop 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 shutdown_connections 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe8c4066f30 0x7fe8c40693e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8e41596d0 0x7fe8e415f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8e40ae620 0x7fe8e414e1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 --2- 172.21.15.123:0/1989810908 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8e40caea0 0x7fe8e416da80 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 >> 172.21.15.123:0/1989810908 conn(0x7fe8e4063a10 msgr2=0x7fe8e414f110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 shutdown_connections 2023-10-06T04:32:49.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.257+0000 7fe8e8f38700 1 -- 172.21.15.123:0/1989810908 wait complete. 2023-10-06T04:32:49.261 INFO:teuthology.orchestra.run.smithi123.stderr:recovery_fs marked joinable; MDS may join as newly active. 2023-10-06T04:32:49.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:49.398 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.396+0000 7f4c0af8d700 1 Processor -- start 2023-10-06T04:32:49.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.396+0000 7f4c0af8d700 1 -- start start 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c0af8d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4c04068c90 0x7f4c0406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c0af8d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f4c0406e970 0x7f4c04158f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c0af8d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c04069a30 con 0x7f4c040688c0 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c0af8d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c0406a890 con 0x7f4c04068c90 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c0af8d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c04069c80 con 0x7f4c0406e970 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4c04068c90 0x7f4c0406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4c04068c90 0x7f4c0406e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34028/0 (socket says 172.21.15.123:34028) 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 -- 172.21.15.123:0/2336403068 learned_addr learned my addr 172.21.15.123:0/2336403068 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c0952a700 1 --2- 172.21.15.123:0/2336403068 >> v2:172.21.15.177:3300/0 conn(0x7f4c0406e970 0x7f4c04158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:49.401 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 -- 172.21.15.123:0/2336403068 >> v2:172.21.15.177:3300/0 conn(0x7f4c0406e970 msgr2=0x7f4c04158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 --2- 172.21.15.123:0/2336403068 >> v2:172.21.15.177:3300/0 conn(0x7f4c0406e970 0x7f4c04158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 -- 172.21.15.123:0/2336403068 >> v1:172.21.15.177:6789/0 conn(0x7f4c040688c0 legacy=0x7f4c040c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 -- 172.21.15.123:0/2336403068 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c04069f60 con 0x7f4c04068c90 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c03fff700 1 --2- 172.21.15.123:0/2336403068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c0406e430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4c040ca7a0 tx=0x7f4bf0009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4429db951f780496 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.397+0000 7f4c037fe700 1 -- 172.21.15.123:0/2336403068 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000b820 con 0x7f4c04068c90 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c037fe700 1 -- 172.21.15.123:0/2336403068 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf000b980 con 0x7f4c04068c90 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c037fe700 1 -- 172.21.15.123:0/2336403068 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf000bd20 con 0x7f4c04068c90 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 -- 172.21.15.123:0/2336403068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 msgr2=0x7f4c0406e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/2336403068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c0406e430 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7f4c040ca7a0 tx=0x7f4bf0009250 comp rx=0 tx=0).stop 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 -- 172.21.15.123:0/2336403068 shutdown_connections 2023-10-06T04:32:49.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/2336403068 >> v2:172.21.15.177:3300/0 conn(0x7f4c0406e970 0x7f4c04158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/2336403068 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c0406e430 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 -- 172.21.15.123:0/2336403068 >> 172.21.15.123:0/2336403068 conn(0x7f4c040c9c90 msgr2=0x7f4c0406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:49.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 -- 172.21.15.123:0/2336403068 shutdown_connections 2023-10-06T04:32:49.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.398+0000 7f4c0af8d700 1 -- 172.21.15.123:0/2336403068 wait complete. 2023-10-06T04:32:49.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.399+0000 7f4c0af8d700 1 Processor -- start 2023-10-06T04:32:49.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.399+0000 7f4c0af8d700 1 -- start start 2023-10-06T04:32:49.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c0af8d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c040688c0 0x7f4c04156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c0af8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c04169c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c0af8d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0406e970 0x7f4c0416d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c0af8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c04156c70 con 0x7f4c04068c90 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c0af8d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c0415acb0 con 0x7f4c0406e970 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c0af8d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c040ad6a0 con 0x7f4c040688c0 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c03fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c04169c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c03fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c04169c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:34040/0 (socket says 172.21.15.123:34040) 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c03fff700 1 -- 172.21.15.123:0/3617150323 learned_addr learned my addr 172.21.15.123:0/3617150323 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c08d29700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c040688c0 0x7f4c04156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:49.406 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.400+0000 7f4c0952a700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0406e970 0x7f4c0416d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c03fff700 1 -- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c040688c0 msgr2=0x7f4c04156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c03fff700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c040688c0 0x7f4c04156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c03fff700 1 -- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0406e970 msgr2=0x7f4c0416d210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c03fff700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0406e970 0x7f4c0416d210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c03fff700 1 -- 172.21.15.123:0/3617150323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf00094f0 con 0x7f4c04068c90 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c03fff700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c04169c10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c0406d230 tx=0x7f4bf0016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c00ff9700 1 -- 172.21.15.123:0/3617150323 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf0011cc0 con 0x7f4c04068c90 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c00ff9700 1 -- 172.21.15.123:0/3617150323 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf0020070 con 0x7f4c04068c90 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c040acdb0 con 0x7f4c04068c90 2023-10-06T04:32:49.407 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c00ff9700 1 -- 172.21.15.123:0/3617150323 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf001b6a0 con 0x7f4c04068c90 2023-10-06T04:32:49.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.401+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c040ad330 con 0x7f4c04068c90 2023-10-06T04:32:49.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.402+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c0406cc70 con 0x7f4c04068c90 2023-10-06T04:32:49.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.402+0000 7f4c00ff9700 1 -- 172.21.15.123:0/3617150323 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4bf0022030 con 0x7f4c04068c90 2023-10-06T04:32:49.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.403+0000 7f4c00ff9700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f60 0x7f4be8069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:49.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.403+0000 7f4c00ff9700 1 -- 172.21.15.123:0/3617150323 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4bf00866c0 con 0x7f4c04068c90 2023-10-06T04:32:49.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.403+0000 7f4c08d29700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f60 0x7f4be8069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:49.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.403+0000 7f4c08d29700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f60 0x7f4be8069410 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8003a30 tx=0x7f4bf8007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:49.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.407+0000 7f4c00ff9700 1 -- 172.21.15.123:0/3617150323 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4bf00535b0 con 0x7f4c04068c90 2023-10-06T04:32:49.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.635+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4c0406cc70 con 0x7f4c04068c90 2023-10-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.635+0000 7f4c00ff9700 1 -- 172.21.15.123:0/3617150323 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 23 v23) v1 ==== 94+0+5671 (secure 0 0 0) 0x7f4bf0056bd0 con 0x7f4c04068c90 2023-10-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:49.638 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":23,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":23,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:49.250452+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:replay","state_seq":1,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f60 msgr2=0x7f4be8069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f60 0x7f4be8069410 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f4bf8003a30 tx=0x7f4bf8007ea0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 msgr2=0x7f4c04169c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c04169c10 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7f4c0406d230 tx=0x7f4bf0016000 comp rx=0 tx=0).stop 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 shutdown_connections 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4be8066f60 0x7f4be8069410 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4c0406e970 0x7f4c0416d210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4c040688c0 0x7f4c04156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 --2- 172.21.15.123:0/3617150323 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4c04068c90 0x7f4c04169c10 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 >> 172.21.15.123:0/3617150323 conn(0x7f4c040c9c90 msgr2=0x7f4c041575f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 shutdown_connections 2023-10-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:49.637+0000 7f4c0af8d700 1 -- 172.21.15.123:0/3617150323 wait complete. 2023-10-06T04:32:49.641 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 23 2023-10-06T04:32:50.053 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 23, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:32:37.297331+0000', 'modified': '2023-10-06T04:32:49.250452+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5295': {'gid': 5295, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:replay', 'state_seq': 1, 'addr': '172.21.15.123:6836/571369576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 571369576}, {'type': 'v1', 'addr': '172.21.15.123:6836', 'nonce': 571369576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:32:50.053 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5295:up:replay 2023-10-06T04:32:51.054 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:51.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.185+0000 7feaa9ab6700 1 Processor -- start 2023-10-06T04:32:51.186 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.185+0000 7feaa9ab6700 1 -- start start 2023-10-06T04:32:51.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.185+0000 7feaa9ab6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feaa4068c90 0x7feaa406e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:51.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa9ab6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feaa406e970 0x7feaa4158f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:51.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa9ab6700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feaa4069a30 con 0x7feaa40688c0 2023-10-06T04:32:51.188 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa9ab6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa406a890 con 0x7feaa4068c90 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa9ab6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feaa4069c80 con 0x7feaa406e970 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7fea9bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feaa4068c90 0x7feaa406e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa37fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7feaa40688c0 0x7feaa40c6dd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36648/0 (socket says 172.21.15.123:36648) 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa37fe700 1 -- 172.21.15.123:0/435977757 learned_addr learned my addr 172.21.15.123:0/435977757 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa3fff700 1 --2- 172.21.15.123:0/435977757 >> v2:172.21.15.177:3300/0 conn(0x7feaa406e970 0x7feaa4158f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7fea9bfff700 1 -- 172.21.15.123:0/435977757 >> v2:172.21.15.177:3300/0 conn(0x7feaa406e970 msgr2=0x7feaa4158f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7fea9bfff700 1 --2- 172.21.15.123:0/435977757 >> v2:172.21.15.177:3300/0 conn(0x7feaa406e970 0x7feaa4158f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7fea9bfff700 1 -- 172.21.15.123:0/435977757 >> v1:172.21.15.123:6789/0 conn(0x7feaa40688c0 legacy=0x7feaa40c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa2ffd700 1 -- 172.21.15.123:0/435977757 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1045635028 0 0) 0x7feaa4069a30 con 0x7feaa40688c0 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7fea9bfff700 1 -- 172.21.15.123:0/435977757 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feaa4069f60 con 0x7feaa4068c90 2023-10-06T04:32:51.189 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa3fff700 1 --2- 172.21.15.123:0/435977757 >> v2:172.21.15.177:3300/0 conn(0x7feaa406e970 0x7feaa4158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7fea9bfff700 1 --2- 172.21.15.123:0/435977757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa406e430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feaa40ca7a0 tx=0x7fea8c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b0e6e06747ae7d9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa2ffd700 1 -- 172.21.15.123:0/435977757 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00b6b0 con 0x7feaa4068c90 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa2ffd700 1 -- 172.21.15.123:0/435977757 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c00b810 con 0x7feaa4068c90 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.186+0000 7feaa2ffd700 1 -- 172.21.15.123:0/435977757 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c00bbb0 con 0x7feaa4068c90 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 -- 172.21.15.123:0/435977757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 msgr2=0x7feaa406e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/435977757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa406e430 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7feaa40ca7a0 tx=0x7fea8c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 -- 172.21.15.123:0/435977757 shutdown_connections 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/435977757 >> v2:172.21.15.177:3300/0 conn(0x7feaa406e970 0x7feaa4158f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/435977757 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa406e430 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 -- 172.21.15.123:0/435977757 >> 172.21.15.123:0/435977757 conn(0x7feaa40c9c90 msgr2=0x7feaa406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 -- 172.21.15.123:0/435977757 shutdown_connections 2023-10-06T04:32:51.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.187+0000 7feaa9ab6700 1 -- 172.21.15.123:0/435977757 wait complete. 2023-10-06T04:32:51.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.188+0000 7feaa9ab6700 1 Processor -- start 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.188+0000 7feaa9ab6700 1 -- start start 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7feaa9ab6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa40688c0 0x7feaa4159980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7feaa9ab6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa40b1900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7feaa9ab6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa406e970 0x7feaa415a260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7feaa9ab6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa4159ec0 con 0x7feaa4068c90 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7feaa9ab6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feaa40c4970 con 0x7feaa40688c0 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7feaa9ab6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feaa41594f0 con 0x7feaa406e970 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa40b1900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa40b1900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33220/0 (socket says 172.21.15.123:33220) 2023-10-06T04:32:51.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 -- 172.21.15.123:0/2371916203 learned_addr learned my addr 172.21.15.123:0/2371916203 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 -- 172.21.15.123:0/2371916203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa406e970 msgr2=0x7feaa415a260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa406e970 0x7feaa415a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 -- 172.21.15.123:0/2371916203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa40688c0 msgr2=0x7feaa4159980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa40688c0 0x7feaa4159980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 -- 172.21.15.123:0/2371916203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea8c009420 con 0x7feaa4068c90 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9bfff700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa40b1900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00b0e0 tx=0x7fea8c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.189+0000 7fea9b7fe700 1 -- 172.21.15.123:0/2371916203 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c011420 con 0x7feaa4068c90 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.190+0000 7fea9b7fe700 1 -- 172.21.15.123:0/2371916203 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea8c011580 con 0x7feaa4068c90 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.190+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feaa4158ee0 con 0x7feaa4068c90 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.190+0000 7fea9b7fe700 1 -- 172.21.15.123:0/2371916203 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea8c011920 con 0x7feaa4068c90 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.190+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feaa4159180 con 0x7feaa4068c90 2023-10-06T04:32:51.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.191+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feaa406cc70 con 0x7feaa4068c90 2023-10-06T04:32:51.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.191+0000 7fea9b7fe700 1 -- 172.21.15.123:0/2371916203 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fea8c011d10 con 0x7feaa4068c90 2023-10-06T04:32:51.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.192+0000 7fea9b7fe700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fea84066f30 0x7fea840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:51.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.192+0000 7fea9b7fe700 1 -- 172.21.15.123:0/2371916203 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fea8c0863f0 con 0x7feaa4068c90 2023-10-06T04:32:51.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.192+0000 7feaa37fe700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fea84066f30 0x7fea840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:51.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.192+0000 7feaa37fe700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fea84066f30 0x7fea840693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea940011e0 tx=0x7fea940079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:51.197 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.195+0000 7fea9b7fe700 1 -- 172.21.15.123:0/2371916203 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fea8c053440 con 0x7feaa4068c90 2023-10-06T04:32:51.419 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.418+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7feaa406a240 con 0x7feaa4068c90 2023-10-06T04:32:51.420 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.419+0000 7fea9b7fe700 1 -- 172.21.15.123:0/2371916203 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 25 v25) v1 ==== 94+0+5671 (secure 0 0 0) 0x7fea8c056a60 con 0x7feaa4068c90 2023-10-06T04:32:51.421 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:51.421 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":25,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":25,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:51.262428+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:rejoin","state_seq":6,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:32:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fea84066f30 msgr2=0x7fea840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fea84066f30 0x7fea840693e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fea940011e0 tx=0x7fea940079d0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 msgr2=0x7feaa40b1900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa40b1900 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7fea8c00b0e0 tx=0x7fea8c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 shutdown_connections 2023-10-06T04:32:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fea84066f30 0x7fea840693e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feaa406e970 0x7feaa415a260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feaa40688c0 0x7feaa4159980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 --2- 172.21.15.123:0/2371916203 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feaa4068c90 0x7feaa40b1900 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:51.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 >> 172.21.15.123:0/2371916203 conn(0x7feaa40c9c90 msgr2=0x7feaa40c1830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 shutdown_connections 2023-10-06T04:32:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:51.421+0000 7feaa9ab6700 1 -- 172.21.15.123:0/2371916203 wait complete. 2023-10-06T04:32:51.424 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 25 2023-10-06T04:32:51.836 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 25, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:32:37.297331+0000', 'modified': '2023-10-06T04:32:51.262428+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5295': {'gid': 5295, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:rejoin', 'state_seq': 6, 'addr': '172.21.15.123:6836/571369576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 571369576}, {'type': 'v1', 'addr': '172.21.15.123:6836', 'nonce': 571369576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:32:51.837 WARNING:tasks.cephfs.filesystem:Unhealthy mds state gid_5295:up:rejoin 2023-10-06T04:32:52.837 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:52.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.966+0000 7f34608bc700 1 Processor -- start 2023-10-06T04:32:52.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.966+0000 7f34608bc700 1 -- start start 2023-10-06T04:32:52.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.967+0000 7f34608bc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c1560c0 0x7f345c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.967+0000 7f34608bc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f345c06e960 0x7f345c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.967+0000 7f34608bc700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f345c0c8580 con 0x7f345c06f080 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.967+0000 7f34608bc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f345c0c8860 con 0x7f345c06e960 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.967+0000 7f34608bc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c8b40 con 0x7f345c1560c0 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345ad9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f345c06e960 0x7f345c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f3459d9b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f345c06f080 0x7f345c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36658/0 (socket says 172.21.15.123:36658) 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f3459d9b700 1 -- 172.21.15.123:0/917895899 learned_addr learned my addr 172.21.15.123:0/917895899 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345a59c700 1 --2- 172.21.15.123:0/917895899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c1560c0 0x7f345c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345ad9d700 1 -- 172.21.15.123:0/917895899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c1560c0 msgr2=0x7f345c0c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345ad9d700 1 --2- 172.21.15.123:0/917895899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c1560c0 0x7f345c0c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:52.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345ad9d700 1 -- 172.21.15.123:0/917895899 >> v1:172.21.15.123:6789/0 conn(0x7f345c06f080 legacy=0x7f345c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:52.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345ad9d700 1 -- 172.21.15.123:0/917895899 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f345c0c8e20 con 0x7f345c06e960 2023-10-06T04:32:52.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345959a700 1 -- 172.21.15.123:0/917895899 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3032114113 0 0) 0x7f345c0c8580 con 0x7f345c06f080 2023-10-06T04:32:52.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345ad9d700 1 --2- 172.21.15.123:0/917895899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c0ada60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3454009e60 tx=0x7f3454009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d6aa8bdf5305d86 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:52.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345959a700 1 -- 172.21.15.123:0/917895899 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345400e070 con 0x7f345c06e960 2023-10-06T04:32:52.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345959a700 1 -- 172.21.15.123:0/917895899 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345400bc30 con 0x7f345c06e960 2023-10-06T04:32:52.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.968+0000 7f345959a700 1 -- 172.21.15.123:0/917895899 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3454012430 con 0x7f345c06e960 2023-10-06T04:32:52.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 -- 172.21.15.123:0/917895899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 msgr2=0x7f345c0ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 --2- 172.21.15.123:0/917895899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c0ada60 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f3454009e60 tx=0x7f3454009780 comp rx=0 tx=0).stop 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 -- 172.21.15.123:0/917895899 shutdown_connections 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 --2- 172.21.15.123:0/917895899 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c0ada60 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 --2- 172.21.15.123:0/917895899 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c1560c0 0x7f345c0c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 -- 172.21.15.123:0/917895899 >> 172.21.15.123:0/917895899 conn(0x7f345c0cda80 msgr2=0x7f345c0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 -- 172.21.15.123:0/917895899 shutdown_connections 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.969+0000 7f34608bc700 1 -- 172.21.15.123:0/917895899 wait complete. 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.970+0000 7f34608bc700 1 Processor -- start 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.970+0000 7f34608bc700 1 -- start start 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c1533d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:52.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c06f080 0x7f345c15afa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f345c1560c0 0x7f345c068770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c5410 con 0x7f345c06e960 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c4800 con 0x7f345c1560c0 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f345c0c4ae0 con 0x7f345c06f080 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c1533d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c1533d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33248/0 (socket says 172.21.15.123:33248) 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 -- 172.21.15.123:0/1562011038 learned_addr learned my addr 172.21.15.123:0/1562011038 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 -- 172.21.15.123:0/1562011038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c06f080 msgr2=0x7f345c15afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c06f080 0x7f345c15afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 -- 172.21.15.123:0/1562011038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f345c1560c0 msgr2=0x7f345c068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f345c1560c0 0x7f345c068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:52.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 -- 172.21.15.123:0/1562011038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3454009a20 con 0x7f345c06e960 2023-10-06T04:32:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f345a59c700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c1533d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f345000d890 tx=0x7f3450008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f344a7fc700 1 -- 172.21.15.123:0/1562011038 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345000b600 con 0x7f345c06e960 2023-10-06T04:32:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f344a7fc700 1 -- 172.21.15.123:0/1562011038 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f345000b760 con 0x7f345c06e960 2023-10-06T04:32:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f345c0c4dc0 con 0x7f345c06e960 2023-10-06T04:32:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f344a7fc700 1 -- 172.21.15.123:0/1562011038 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f345000bae0 con 0x7f345c06e960 2023-10-06T04:32:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.971+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f345c153910 con 0x7f345c06e960 2023-10-06T04:32:52.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.972+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f345c0cde80 con 0x7f345c06e960 2023-10-06T04:32:52.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.973+0000 7f344a7fc700 1 -- 172.21.15.123:0/1562011038 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f345001e460 con 0x7f345c06e960 2023-10-06T04:32:52.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.973+0000 7f344a7fc700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f343c066f30 0x7f343c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:52.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.973+0000 7f344a7fc700 1 -- 172.21.15.123:0/1562011038 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f3450084f60 con 0x7f345c06e960 2023-10-06T04:32:52.977 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.973+0000 7f3459d9b700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f343c066f30 0x7f343c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:52.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.974+0000 7f3459d9b700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f343c066f30 0x7f343c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f344c0011e0 tx=0x7f344c0079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:52.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:52.977+0000 7f344a7fc700 1 -- 172.21.15.123:0/1562011038 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3450051e50 con 0x7f345c06e960 2023-10-06T04:32:53.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.198+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f345c04bb80 con 0x7f345c06e960 2023-10-06T04:32:53.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.199+0000 7f344a7fc700 1 -- 172.21.15.123:0/1562011038 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5671 (secure 0 0 0) 0x7f3450055470 con 0x7f345c06e960 2023-10-06T04:32:53.200 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:53.200 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:32:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f343c066f30 msgr2=0x7f343c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f343c066f30 0x7f343c0693e0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f344c0011e0 tx=0x7f344c0079d0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 msgr2=0x7f345c1533d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c1533d0 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f345000d890 tx=0x7f3450008040 comp rx=0 tx=0).stop 2023-10-06T04:32:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 shutdown_connections 2023-10-06T04:32:53.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f343c066f30 0x7f343c0693e0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f345c06e960 0x7f345c1533d0 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f345c1560c0 0x7f345c068770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 --2- 172.21.15.123:0/1562011038 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f345c06f080 0x7f345c15afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 >> 172.21.15.123:0/1562011038 conn(0x7f345c0cda80 msgr2=0x7f345c155330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 shutdown_connections 2023-10-06T04:32:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.201+0000 7f34608bc700 1 -- 172.21.15.123:0/1562011038 wait complete. 2023-10-06T04:32:53.204 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 26 2023-10-06T04:32:53.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 26, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-10-06T04:32:37.297331+0000', 'modified': '2023-10-06T04:32:52.264088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5295}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5295': {'gid': 5295, 'name': 'a', 'rank': 0, 'incarnation': 23, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.123:6836/571369576', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.123:6834', 'nonce': 571369576}, {'type': 'v1', 'addr': '172.21.15.123:6836', 'nonce': 571369576}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138320759226367, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [6], 'metadata_pool': 7, 'enabled': True, 'fs_name': 'recovery_fs', 'balancer': '', 'standby_count_wanted': 0} 2023-10-06T04:32:53.617 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-10-06T04:32:53.617 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-10-06T04:32:53.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.737+0000 7f2513fda700 1 Processor -- start 2023-10-06T04:32:53.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.737+0000 7f2513fda700 1 -- start start 2023-10-06T04:32:53.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2513fda700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f250c157140 0x7f250c06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2513fda700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2513fda700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f250c06aa70 con 0x7f250c0c3840 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2513fda700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f250c0cb810 con 0x7f250c157140 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2513fda700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0abf70 con 0x7f250c157510 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2511d76700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f250c157140 0x7f250c06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2511d76700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f250c157140 0x7f250c06bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33250/0 (socket says 172.21.15.123:33250) 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2511d76700 1 -- 172.21.15.123:0/773272636 learned_addr learned my addr 172.21.15.123:0/773272636 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.738+0000 7f2511575700 1 --2- 172.21.15.123:0/773272636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2511d76700 1 -- 172.21.15.123:0/773272636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 msgr2=0x7f250c0aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2511d76700 1 --2- 172.21.15.123:0/773272636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0aba30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2511d76700 1 -- 172.21.15.123:0/773272636 >> v1:172.21.15.177:6789/0 conn(0x7f250c0c3840 legacy=0x7f250c0cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.742 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2511d76700 1 -- 172.21.15.123:0/773272636 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f250c06a4b0 con 0x7f250c157140 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2511d76700 1 --2- 172.21.15.123:0/773272636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c157140 0x7f250c06bf70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0099b0 tx=0x7f24fc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab076cde3cddcecb server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2510d74700 1 -- 172.21.15.123:0/773272636 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00b820 con 0x7f250c157140 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2510d74700 1 -- 172.21.15.123:0/773272636 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc00b980 con 0x7f250c157140 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2510d74700 1 -- 172.21.15.123:0/773272636 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc00bd20 con 0x7f250c157140 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2513fda700 1 -- 172.21.15.123:0/773272636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c157140 msgr2=0x7f250c06bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2513fda700 1 --2- 172.21.15.123:0/773272636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c157140 0x7f250c06bf70 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f24fc0099b0 tx=0x7f24fc009250 comp rx=0 tx=0).stop 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2513fda700 1 -- 172.21.15.123:0/773272636 shutdown_connections 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2513fda700 1 --2- 172.21.15.123:0/773272636 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c157140 0x7f250c06bf70 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2513fda700 1 --2- 172.21.15.123:0/773272636 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2513fda700 1 -- 172.21.15.123:0/773272636 >> 172.21.15.123:0/773272636 conn(0x7f250c0c9d00 msgr2=0x7f250c06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.743 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.739+0000 7f2513fda700 1 -- 172.21.15.123:0/773272636 shutdown_connections 2023-10-06T04:32:53.744 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.740+0000 7f2513fda700 1 -- 172.21.15.123:0/773272636 wait complete. 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.741+0000 7f2513fda700 1 Processor -- start 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.741+0000 7f2513fda700 1 -- start start 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2513fda700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c0c3840 0x7f250c14e670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2513fda700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250c157140 0x7f250c0c5570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2513fda700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0c2320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2513fda700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0c50e0 con 0x7f250c0c3840 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2513fda700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f250c0c4ad0 con 0x7f250c157140 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2513fda700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f250c1550c0 con 0x7f250c157510 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c0c3840 0x7f250c14e670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c0c3840 0x7f250c14e670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33258/0 (socket says 172.21.15.123:33258) 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 -- 172.21.15.123:0/2056696857 learned_addr learned my addr 172.21.15.123:0/2056696857 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2512577700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0c2320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511575700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250c157140 0x7f250c0c5570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 -- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 msgr2=0x7f250c0c2320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0c2320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 -- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250c157140 msgr2=0x7f250c0c5570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250c157140 0x7f250c0c5570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 -- 172.21.15.123:0/2056696857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24fc0094f0 con 0x7f250c0c3840 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.742+0000 7f2511d76700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c0c3840 0x7f250c14e670 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009290 tx=0x7f24fc016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.743+0000 7f2501ffb700 1 -- 172.21.15.123:0/2056696857 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc011ca0 con 0x7f250c0c3840 2023-10-06T04:32:53.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.743+0000 7f2501ffb700 1 -- 172.21.15.123:0/2056696857 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc020070 con 0x7f250c0c3840 2023-10-06T04:32:53.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.743+0000 7f2501ffb700 1 -- 172.21.15.123:0/2056696857 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc01b6a0 con 0x7f250c0c3840 2023-10-06T04:32:53.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.743+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f250c0c4d70 con 0x7f250c0c3840 2023-10-06T04:32:53.748 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.743+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c14ebb0 con 0x7f250c0c3840 2023-10-06T04:32:53.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.746+0000 7f251d2eeb80 1 Processor -- start 2023-10-06T04:32:53.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.746+0000 7f251d2eeb80 1 -- start start 2023-10-06T04:32:53.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.747+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4b05a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.747+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4fa370 0x55888f4b0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.747+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4f6210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.747+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f3c9310 con 0x55888f4fa370 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.747+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f406130 con 0x55888f4d8790 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.747+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55888f403af0 con 0x55888f4b1020 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2501ffb700 1 -- 172.21.15.123:0/2056696857 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24fc022030 con 0x7f250c0c3840 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511575700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4f6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2512577700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4b05a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511575700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4f6210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3301/0 says I am v2:172.21.15.123:48610/0 (socket says 172.21.15.123:48610) 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2512577700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4b05a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.177:3300/0 says I am v2:172.21.15.123:34984/0 (socket says 172.21.15.123:34984) 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511d76700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4fa370 0x55888f4b0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.752 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511575700 1 -- 172.21.15.123:0/4109842166 learned_addr learned my addr 172.21.15.123:0/4109842166 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511d76700 1 -- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 msgr2=0x55888f4f6210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511d76700 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4f6210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511d76700 1 -- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 msgr2=0x55888f4b05a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511d76700 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4b05a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511d76700 1 -- 172.21.15.123:0/4109842166 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55888f205790 con 0x55888f4fa370 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2501ffb700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24f0066f30 0x7f24f00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2512577700 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511575700 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2512577700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24f0066f30 0x7f24f00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2501ffb700 1 -- 172.21.15.123:0/2056696857 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f24fc086750 con 0x7f250c0c3840 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f2511d76700 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4fa370 0x55888f4b0ae0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f24fc084850 tx=0x7f24fc0847b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdeff1ebb02a823d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f24eeffd700 1 -- 172.21.15.123:0/4109842166 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc057030 con 0x55888f4fa370 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f24eeffd700 1 -- 172.21.15.123:0/4109842166 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc056da0 con 0x55888f4fa370 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.748+0000 7f24eeffd700 1 -- 172.21.15.123:0/4109842166 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24fc056bc0 con 0x55888f4fa370 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 -- 172.21.15.123:0/4109842166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4fa370 msgr2=0x55888f4b0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f2512577700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24f0066f30 0x7f24f00693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2504007fd0 tx=0x7f2504009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4fa370 0x55888f4b0ae0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7f24fc084850 tx=0x7f24fc0847b0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 -- 172.21.15.123:0/4109842166 shutdown_connections 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4f6210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4b05a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/4109842166 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4fa370 0x55888f4b0ae0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 -- 172.21.15.123:0/4109842166 >> 172.21.15.123:0/4109842166 conn(0x55888f4f9890 msgr2=0x55888f4f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 -- 172.21.15.123:0/4109842166 shutdown_connections 2023-10-06T04:32:53.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.749+0000 7f251d2eeb80 1 -- 172.21.15.123:0/4109842166 wait complete. 2023-10-06T04:32:53.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.750+0000 7f251d2eeb80 1 -- start start 2023-10-06T04:32:53.755 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.750+0000 7f251d2eeb80 1 Processor -- start 2023-10-06T04:32:53.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4af2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4af7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4fa370 0x55888f4d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f3ff8e0 con 0x55888f4d8790 2023-10-06T04:32:53.758 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f3ab980 con 0x55888f4fa370 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55888f3ab620 con 0x55888f4b1020 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 10 client.0 _ll_register_callbacks cb 0x55888f20ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4af7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4af7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33272/0 (socket says 172.21.15.123:33272) 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 -- 172.21.15.123:0/3641945732 learned_addr learned my addr 172.21.15.123:0/3641945732 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511d76700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4af2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2512577700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4fa370 0x55888f4d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 -- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 msgr2=0x55888f4af2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4af2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 -- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4fa370 msgr2=0x55888f4d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.759 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4fa370 0x55888f4d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 -- 172.21.15.123:0/3641945732 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250400f040 con 0x55888f4d8790 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511d76700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f2511575700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4af7e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f25080126b0 tx=0x7f2508008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce98ae2d5947060f server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f24e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f24e7fff700 1 -- 172.21.15.123:0/3641945732 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508014690 con 0x55888f4d8790 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f24e7fff700 1 -- 172.21.15.123:0/3641945732 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f25080147f0 con 0x55888f4d8790 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.751+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55888f4a3e50 con 0x55888f4d8790 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f24e7fff700 1 -- 172.21.15.123:0/3641945732 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2508014bc0 con 0x55888f4d8790 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 10 client.5574 fetch_fsmap learned FSMap version 26 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x7f250c14ebb0 con 0x55888f4d8790 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f24e7fff700 1 -- 172.21.15.123:0/3641945732 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55888f4a3e50 con 0x55888f4d8790 2023-10-06T04:32:53.760 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f24e7fff700 1 -- 172.21.15.123:0/3641945732 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 3659+0+0 (secure 0 0 0) 0x7f2508014d20 con 0x55888f4d8790 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 10 client.5574 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 10 client.5574 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 20 client.5574 populate_metadata read hostname 'smithi123' 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f50d5d0 0x55888f50d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 4 client.5574 mds_command: new command op to 5295 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 --> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55888f480c20 con 0x55888f50d5d0 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f2511d76700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f50d5d0 0x55888f50d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f24e7fff700 10 client.5574 ms_handle_connect on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.752+0000 7f2511d76700 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f50d5d0 0x55888f50d9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3905d76bf8ed4702 server_cookie=d7ba02be8b2fb456 in_seq=0 out_seq=0 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f24e7fff700 1 -- 172.21.15.123:0/3641945732 <== mds.0 v2:172.21.15.123:6834/571369576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x55888f480c20 con 0x55888f50d5d0 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f24e7fff700 10 client.5574 handle_command_reply: tid=0 2023-10-06T04:32:53.761 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 client.5574 shutdown 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 msgr2=0x55888f4af7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4af7e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f25080126b0 tx=0x7f2508008040 comp rx=0 tx=0).stop 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 shutdown_connections 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f50d5d0 0x55888f50d9a0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4fa370 0x55888f4d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f24e7fff700 0 client.5574 ms_handle_reset on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4af2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/3641945732 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4af7e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 >> 172.21.15.123:0/3641945732 conn(0x55888f4f9890 msgr2=0x55888f4b1500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 shutdown_connections 2023-10-06T04:32:53.762 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.753+0000 7f251d2eeb80 1 -- 172.21.15.123:0/3641945732 wait complete. 2023-10-06T04:32:53.763 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.754+0000 7f251d2eeb80 20 client.5574 trim_cache size 0 max 16384 2023-10-06T04:32:53.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.770+0000 7f251d2eeb80 1 Processor -- start 2023-10-06T04:32:53.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.770+0000 7f251d2eeb80 1 -- start start 2023-10-06T04:32:53.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4f9890 0x55888f4d7c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4b1020 0x55888f4af840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4a50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f2971e0 con 0x55888f4b1020 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f4afd80 con 0x55888f4d8790 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55888f50a8b0 con 0x55888f4f9890 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4b1020 0x55888f4af840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4b1020 0x55888f4af840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33276/0 (socket says 172.21.15.123:33276) 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 -- 172.21.15.123:0/1813289469 learned_addr learned my addr 172.21.15.123:0/1813289469 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511d76700 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4f9890 0x55888f4d7c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.774 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 -- 172.21.15.123:0/1813289469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4f9890 msgr2=0x55888f4d7c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4f9890 0x55888f4d7c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 -- 172.21.15.123:0/1813289469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 msgr2=0x55888f4a50c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4a50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 -- 172.21.15.123:0/1813289469 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55888f2f20a0 con 0x55888f4b1020 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f2511575700 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4b1020 0x55888f4af840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2508014010 tx=0x7f250801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0095ee92e496c1c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f24ef7fe700 1 -- 172.21.15.123:0/1813289469 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508015590 con 0x55888f4b1020 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f24ef7fe700 1 -- 172.21.15.123:0/1813289469 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2508014690 con 0x55888f4b1020 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.771+0000 7f24ef7fe700 1 -- 172.21.15.123:0/1813289469 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2508000e30 con 0x55888f4b1020 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 -- 172.21.15.123:0/1813289469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4b1020 msgr2=0x55888f4af840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4b1020 0x55888f4af840 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f2508014010 tx=0x7f250801a000 comp rx=0 tx=0).stop 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 -- 172.21.15.123:0/1813289469 shutdown_connections 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4d8790 0x55888f4a50c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4f9890 0x55888f4d7c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/1813289469 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4b1020 0x55888f4af840 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 -- 172.21.15.123:0/1813289469 >> 172.21.15.123:0/1813289469 conn(0x55888f4d81f0 msgr2=0x55888f4f9c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 -- 172.21.15.123:0/1813289469 shutdown_connections 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 -- 172.21.15.123:0/1813289469 wait complete. 2023-10-06T04:32:53.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 -- start start 2023-10-06T04:32:53.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.772+0000 7f251d2eeb80 1 Processor -- start 2023-10-06T04:32:53.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.783 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4ddf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.784 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f251d2eeb80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4f9890 0x55888f4d5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.784 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f4a5660 con 0x55888f4d8790 2023-10-06T04:32:53.784 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x55888f4d4370 con 0x55888f4f9890 2023-10-06T04:32:53.784 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f251d2eeb80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x55888f4dea60 con 0x55888f4b1020 2023-10-06T04:32:53.784 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f251d2eeb80 10 client.0 _ll_register_callbacks cb 0x55888f20ea00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:32:53.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f2511575700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4ddf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f2511575700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4ddf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33290/0 (socket says 172.21.15.123:33290) 2023-10-06T04:32:53.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.773+0000 7f2511575700 1 -- 172.21.15.123:0/2506904687 learned_addr learned my addr 172.21.15.123:0/2506904687 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:53.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511d76700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511575700 1 -- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 msgr2=0x55888f4d3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2512577700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4f9890 0x55888f4d5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.785 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511575700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4d3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511575700 1 -- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4f9890 msgr2=0x55888f4d5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511575700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4f9890 0x55888f4d5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511575700 1 -- 172.21.15.123:0/2506904687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f250c14ebb0 con 0x55888f4d8790 2023-10-06T04:32:53.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511d76700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4d3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:53.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f2511575700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4ddf70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2508016000 tx=0x7f2508022000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e41753d1deb7180 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f24e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:32:53.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f24e7fff700 1 -- 172.21.15.123:0/2506904687 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25080154a0 con 0x55888f4d8790 2023-10-06T04:32:53.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f24e7fff700 1 -- 172.21.15.123:0/2506904687 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f25080157a0 con 0x55888f4d8790 2023-10-06T04:32:53.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f24fc056da0 con 0x55888f4d8790 2023-10-06T04:32:53.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f24e7fff700 1 -- 172.21.15.123:0/2506904687 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f2508015b70 con 0x55888f4d8790 2023-10-06T04:32:53.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 10 client.5580 fetch_fsmap learned FSMap version 26 2023-10-06T04:32:53.787 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x55888f3ce3c0 con 0x55888f4d8790 2023-10-06T04:32:53.788 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f24e7fff700 1 -- 172.21.15.123:0/2506904687 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24fc056da0 con 0x55888f4d8790 2023-10-06T04:32:53.788 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f24e7fff700 1 -- 172.21.15.123:0/2506904687 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 3659+0+0 (secure 0 0 0) 0x7f2508015cd0 con 0x55888f4d8790 2023-10-06T04:32:53.788 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 10 client.5580 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T04:32:53.788 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 10 client.5580 resolve_mds: resolved name 'a' to daemon mds.a 2023-10-06T04:32:53.788 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 20 client.5580 populate_metadata read hostname 'smithi123' 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f4fa370 0x55888f4b1470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 4 client.5580 mds_command: new command op to 5295 tid=0[{"prefix": "status"}] 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.774+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 --> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55888f3a28d0 con 0x55888f4fa370 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f2511d76700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f4fa370 0x55888f4b1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f24e7fff700 10 client.5580 ms_handle_connect on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f2511d76700 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f4fa370 0x55888f4b1470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ccb20c4b47c55122 server_cookie=83494306f07c0596 in_seq=0 out_seq=0 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f24e7fff700 1 -- 172.21.15.123:0/2506904687 <== mds.0 v2:172.21.15.123:6834/571369576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+336 (crc 0 0 0) 0x55888f3a28d0 con 0x55888f4fa370 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f24e7fff700 10 client.5580 handle_command_reply: tid=0 2023-10-06T04:32:53.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 client.5580 shutdown 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 msgr2=0x55888f4ddf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4ddf70 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7f2508016000 tx=0x7f2508022000 comp rx=0 tx=0).stop 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 shutdown_connections 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x55888f4fa370 0x55888f4b1470 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x55888f4f9890 0x55888f4d5be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x55888f4b1020 0x55888f4d3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 --2- 172.21.15.123:0/2506904687 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x55888f4d8790 0x55888f4ddf70 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f24e7fff700 0 client.5580 ms_handle_reset on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 >> 172.21.15.123:0/2506904687 conn(0x55888f4d81f0 msgr2=0x55888f4d68c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.790 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 shutdown_connections 2023-10-06T04:32:53.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.775+0000 7f251d2eeb80 1 -- 172.21.15.123:0/2506904687 wait complete. 2023-10-06T04:32:53.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f251d2eeb80 20 client.5580 trim_cache size 0 max 16384 2023-10-06T04:32:53.791 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:53.791 INFO:teuthology.orchestra.run.smithi123.stdout: "cluster_fsid": "1cea82d3-d2ef-4e20-8c3e-67296f7df92c", 2023-10-06T04:32:53.791 INFO:teuthology.orchestra.run.smithi123.stdout: "whoami": 0, 2023-10-06T04:32:53.791 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 5295, 2023-10-06T04:32:53.791 INFO:teuthology.orchestra.run.smithi123.stdout: "want_state": "up:active", 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout: "state": "up:active", 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "recovery_fs", 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout: "rank_uptime": 4.5234988820000002, 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout: "mdsmap_epoch": 26, 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch": 42, 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout: "osdmap_epoch_barrier": 42, 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout: "uptime": 19.856516128999999 2023-10-06T04:32:53.792 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:53.793 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24f0066f30 msgr2=0x7f24f00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24f0066f30 0x7f24f00693e0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f2504007fd0 tx=0x7f2504009040 comp rx=0 tx=0).stop 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c0c3840 msgr2=0x7f250c14e670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c0c3840 0x7f250c14e670 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f24fc009290 tx=0x7f24fc016000 comp rx=0 tx=0).stop 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 shutdown_connections 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24f0066f30 0x7f24f00693e0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f250c0c3840 0x7f250c14e670 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f250c157510 0x7f250c0c2320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 --2- 172.21.15.123:0/2056696857 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f250c157140 0x7f250c0c5570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 >> 172.21.15.123:0/2056696857 conn(0x7f250c0c9d00 msgr2=0x7f250c150570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 shutdown_connections 2023-10-06T04:32:53.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.776+0000 7f2513fda700 1 -- 172.21.15.123:0/2056696857 wait complete. 2023-10-06T04:32:53.795 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mds debug_mds 20 2023-10-06T04:32:53.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.913+0000 7efea4e5b700 1 Processor -- start 2023-10-06T04:32:53.915 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.913+0000 7efea4e5b700 1 -- start start 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.914+0000 7efea4e5b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7efea006a860 0x7efea006af90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.914+0000 7efea4e5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 0x7efea0068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.914+0000 7efea4e5b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efea006b9c0 con 0x7efea015a1e0 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.914+0000 7efea4e5b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efea0069aa0 con 0x7efea006a860 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.914+0000 7efea4e5b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea0069d80 con 0x7efea006b4d0 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.914+0000 7efe9e59c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efea015a1e0 0x7efea015a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36662/0 (socket says 172.21.15.123:36662) 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9e59c700 1 -- 172.21.15.123:0/4106970965 learned_addr learned my addr 172.21.15.123:0/4106970965 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9ed9d700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 0x7efea0068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 569508633 0 0) 0x7efea006b9c0 con 0x7efea015a1e0 2023-10-06T04:32:53.919 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/4106970965 >> v2:172.21.15.177:3300/0 conn(0x7efea006a860 0x7efea006af90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe8c003570 con 0x7efea015a1e0 2023-10-06T04:32:53.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1524513295 0 0) 0x7efe8c003570 con 0x7efea015a1e0 2023-10-06T04:32:53.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 msgr2=0x7efea0068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 0x7efea0068670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 >> v2:172.21.15.177:3300/0 conn(0x7efea006a860 msgr2=0x7efea006af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 --2- 172.21.15.123:0/4106970965 >> v2:172.21.15.177:3300/0 conn(0x7efea006a860 0x7efea006af90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea006a060 con 0x7efea015a1e0 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7efe90001470 con 0x7efea015a1e0 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 >> v1:172.21.15.123:6789/0 conn(0x7efea015a1e0 legacy=0x7efea015a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 0x7efe8c003a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efe8c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe8c004d50 0x7efe8c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0069aa0 con 0x7efe8c003fe0 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea0069d80 con 0x7efe8c004d50 2023-10-06T04:32:53.921 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c00ba90 con 0x7efea006b4d0 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efe8c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9e59c700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 0x7efe8c003a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9ed9d700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe8c004d50 0x7efe8c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9dd9b700 1 -- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 msgr2=0x7efe8c003a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea006b4d0 0x7efe8c003a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9dd9b700 1 -- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe8c004d50 msgr2=0x7efe8c0051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe8c004d50 0x7efe8c0051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.915+0000 7efe9dd9b700 1 -- 172.21.15.123:0/4106970965 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c003570 con 0x7efe8c003fe0 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efe8c004810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efe88009c00 tx=0x7efe880091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631ab96cd29dccc3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800e6b0 con 0x7efe8c003fe0 2023-10-06T04:32:53.922 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe8800e810 con 0x7efe8c003fe0 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efe9d59a700 1 -- 172.21.15.123:0/4106970965 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe8800ebd0 con 0x7efe8c003fe0 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efea4e5b700 1 -- 172.21.15.123:0/4106970965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 msgr2=0x7efe8c004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efea4e5b700 1 --2- 172.21.15.123:0/4106970965 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efe8c004810 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7efe88009c00 tx=0x7efe880091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efe9ed9d700 1 -- 172.21.15.123:0/4106970965 reap_dead start 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efea4e5b700 1 -- 172.21.15.123:0/4106970965 shutdown_connections 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efea4e5b700 1 -- 172.21.15.123:0/4106970965 >> 172.21.15.123:0/4106970965 conn(0x7efea00c9ca0 msgr2=0x7efea006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efea4e5b700 1 -- 172.21.15.123:0/4106970965 shutdown_connections 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.916+0000 7efea4e5b700 1 -- 172.21.15.123:0/4106970965 wait complete. 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.917+0000 7efea4e5b700 1 Processor -- start 2023-10-06T04:32:53.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.917+0000 7efea4e5b700 1 -- start start 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efea4e5b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efea0152030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efea4e5b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea00aaa50 0x7efea00c5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efea4e5b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efea00b2010 0x7efea00c6d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efea4e5b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c3850 con 0x7efe8c003fe0 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efea4e5b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efea016ab90 con 0x7efea00b2010 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efea4e5b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efea00c3ac0 con 0x7efea00aaa50 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efe9e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efea0152030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efe9e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efea0152030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33304/0 (socket says 172.21.15.123:33304) 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efe9e59c700 1 -- 172.21.15.123:0/2501359711 learned_addr learned my addr 172.21.15.123:0/2501359711 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:53.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efe9ed9d700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efea00b2010 0x7efea00c6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.918+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea00aaa50 0x7efea00c5230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe9e59c700 1 -- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea00aaa50 msgr2=0x7efea00c5230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe9e59c700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea00aaa50 0x7efea00c5230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe9e59c700 1 -- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efea00b2010 msgr2=0x7efea00c6d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe9e59c700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efea00b2010 0x7efea00c6d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe9e59c700 1 -- 172.21.15.123:0/2501359711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efea00ab270 con 0x7efe8c003fe0 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe9e59c700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efea0152030 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efe90007390 tx=0x7efe90001e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe967fc700 1 -- 172.21.15.123:0/2501359711 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe90018030 con 0x7efe8c003fe0 2023-10-06T04:32:53.927 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe967fc700 1 -- 172.21.15.123:0/2501359711 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe9000f850 con 0x7efe8c003fe0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efea016ae70 con 0x7efe8c003fe0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efe967fc700 1 -- 172.21.15.123:0/2501359711 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe9000fc10 con 0x7efe8c003fe0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.919+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efea00c7280 con 0x7efe8c003fe0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.920+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efea00a98b0 con 0x7efe8c003fe0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.920+0000 7efe967fc700 1 -- 172.21.15.123:0/2501359711 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efe9001b070 con 0x7efe8c003fe0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.921+0000 7efe967fc700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe80066f30 0x7efe800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.921+0000 7efe967fc700 1 -- 172.21.15.123:0/2501359711 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7efe90019070 con 0x7efe8c003fe0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.921+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe80066f30 0x7efe800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:53.928 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.921+0000 7efe9dd9b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe80066f30 0x7efe800693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe880091e0 tx=0x7efe8801a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:53.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:53.925+0000 7efe967fc700 1 -- 172.21.15.123:0/2501359711 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7efe900515d0 con 0x7efe8c003fe0 2023-10-06T04:32:54.110 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.108+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=debug_mds}] v 0) v1 -- 0x7efea00bfd30 con 0x7efe8c003fe0 2023-10-06T04:32:54.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.112+0000 7efe967fc700 1 -- 172.21.15.123:0/2501359711 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{prefix=config set, name=debug_mds}]=0 v11)=0 v11) v1 ==== 108+0+0 (secure 0 0 0) 0x7efe9005a020 con 0x7efe8c003fe0 2023-10-06T04:32:54.116 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe80066f30 msgr2=0x7efe800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe80066f30 0x7efe800693e0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7efe880091e0 tx=0x7efe8801a000 comp rx=0 tx=0).stop 2023-10-06T04:32:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 msgr2=0x7efea0152030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efea0152030 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7efe90007390 tx=0x7efe90001e90 comp rx=0 tx=0).stop 2023-10-06T04:32:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 shutdown_connections 2023-10-06T04:32:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efe80066f30 0x7efe800693e0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.117 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe8c003fe0 0x7efea0152030 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efea00b2010 0x7efea00c6d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 --2- 172.21.15.123:0/2501359711 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efea00aaa50 0x7efea00c5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 >> 172.21.15.123:0/2501359711 conn(0x7efea00c9ca0 msgr2=0x7efea00acce0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 shutdown_connections 2023-10-06T04:32:54.118 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.114+0000 7efea4e5b700 1 -- 172.21.15.123:0/2501359711 wait complete. 2023-10-06T04:32:54.128 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.3:0 scrub start / force,recursive,repair 2023-10-06T04:32:54.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.247+0000 7f41100ae700 1 Processor -- start 2023-10-06T04:32:54.249 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.247+0000 7f41100ae700 1 -- start start 2023-10-06T04:32:54.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f41100ae700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f41081573e0 0x7f41080ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f41100ae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f41080acd90 0x7f41080adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f41100ae700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41080ae960 con 0x7f41080ae500 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f41100ae700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4108157ba0 con 0x7f41081573e0 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f41100ae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41080ad350 con 0x7f41080acd90 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f410de4a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f41081573e0 0x7f41080ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f410de4a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f41081573e0 0x7f41080ac850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33318/0 (socket says 172.21.15.123:33318) 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.248+0000 7f410de4a700 1 -- 172.21.15.123:0/55313300 learned_addr learned my addr 172.21.15.123:0/55313300 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410d649700 1 --2- 172.21.15.123:0/55313300 >> v2:172.21.15.177:3300/0 conn(0x7f41080acd90 0x7f41080adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410de4a700 1 -- 172.21.15.123:0/55313300 >> v2:172.21.15.177:3300/0 conn(0x7f41080acd90 msgr2=0x7f41080adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410de4a700 1 --2- 172.21.15.123:0/55313300 >> v2:172.21.15.177:3300/0 conn(0x7f41080acd90 0x7f41080adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410de4a700 1 -- 172.21.15.123:0/55313300 >> v1:172.21.15.177:6789/0 conn(0x7f41080ae500 legacy=0x7f4108157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410de4a700 1 -- 172.21.15.123:0/55313300 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41080ad630 con 0x7f41081573e0 2023-10-06T04:32:54.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410de4a700 1 --2- 172.21.15.123:0/55313300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41081573e0 0x7f41080ac850 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f41080b2f30 tx=0x7f40f800b4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0bfcc401cf87c94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410ce48700 1 -- 172.21.15.123:0/55313300 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f800dc10 con 0x7f41081573e0 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410ce48700 1 -- 172.21.15.123:0/55313300 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f800dd70 con 0x7f41081573e0 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.249+0000 7f410ce48700 1 -- 172.21.15.123:0/55313300 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80106c0 con 0x7f41081573e0 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 -- 172.21.15.123:0/55313300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41081573e0 msgr2=0x7f41080ac850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 --2- 172.21.15.123:0/55313300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41081573e0 0x7f41080ac850 secure :-1 s=READY pgs=504 cs=0 l=1 rev1=1 crypto rx=0x7f41080b2f30 tx=0x7f40f800b4f0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 -- 172.21.15.123:0/55313300 shutdown_connections 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 --2- 172.21.15.123:0/55313300 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41081573e0 0x7f41080ac850 unknown :-1 s=CLOSED pgs=504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 --2- 172.21.15.123:0/55313300 >> v2:172.21.15.177:3300/0 conn(0x7f41080acd90 0x7f41080adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 -- 172.21.15.123:0/55313300 >> 172.21.15.123:0/55313300 conn(0x7f41080cdb10 msgr2=0x7f41080abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 -- 172.21.15.123:0/55313300 shutdown_connections 2023-10-06T04:32:54.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.250+0000 7f41100ae700 1 -- 172.21.15.123:0/55313300 wait complete. 2023-10-06T04:32:54.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.251+0000 7f41100ae700 1 Processor -- start 2023-10-06T04:32:54.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.251+0000 7f41100ae700 1 -- start start 2023-10-06T04:32:54.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f41100ae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41080acd90 0x7f41080cba00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f41100ae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41080ae500 0x7f41080ca030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f41100ae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41081573e0 0x7f41080ca570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f41100ae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f410809a930 con 0x7f41080acd90 2023-10-06T04:32:54.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f41100ae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41081470c0 con 0x7f41080ae500 2023-10-06T04:32:54.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f41100ae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4108097220 con 0x7f41081573e0 2023-10-06T04:32:54.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41080acd90 0x7f41080cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.257 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41080acd90 0x7f41080cba00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33326/0 (socket says 172.21.15.123:33326) 2023-10-06T04:32:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 -- 172.21.15.123:0/193582605 learned_addr learned my addr 172.21.15.123:0/193582605 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410d649700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41080ae500 0x7f41080ca030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410e64b700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41081573e0 0x7f41080ca570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 -- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41081573e0 msgr2=0x7f41080ca570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41081573e0 0x7f41080ca570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 -- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41080ae500 msgr2=0x7f41080ca030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41080ae500 0x7f41080ca030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 -- 172.21.15.123:0/193582605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40f800b790 con 0x7f41080acd90 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410d649700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41080ae500 0x7f41080ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.252+0000 7f410de4a700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41080acd90 0x7f41080cba00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f410806a1a0 tx=0x7f40f801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.254+0000 7f40fdffb700 1 -- 172.21.15.123:0/193582605 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8018630 con 0x7f41080acd90 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.254+0000 7f40fdffb700 1 -- 172.21.15.123:0/193582605 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f8018790 con 0x7f41080acd90 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.254+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41080cbf40 con 0x7f41080acd90 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.254+0000 7f40fdffb700 1 -- 172.21.15.123:0/193582605 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f8018b50 con 0x7f41080acd90 2023-10-06T04:32:54.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.254+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4108096bd0 con 0x7f41080acd90 2023-10-06T04:32:54.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.256+0000 7f40fdffb700 1 -- 172.21.15.123:0/193582605 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f40f80233d0 con 0x7f41080acd90 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.256+0000 7f40fdffb700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40ec066f30 0x7f40ec0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.256+0000 7f40fdffb700 1 -- 172.21.15.123:0/193582605 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f40f8014070 con 0x7f41080acd90 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.256+0000 7f410d649700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40ec066f30 0x7f40ec0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.257+0000 7f410d649700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40ec066f30 0x7f40ec0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4104006fd0 tx=0x7f4104008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.258+0000 7f41193c2b80 1 Processor -- start 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.258+0000 7f41193c2b80 1 -- start start 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa3ff6c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa4040460 0x560fa3ff7ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa3ff1350 0x560fa3ff17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f41193c2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa3f0f400 con 0x560fa3ff6860 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa3f4c220 con 0x560fa3ff1350 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fa3f49be0 con 0x560fa4040460 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa3ff6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa3ff6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33338/0 (socket says 172.21.15.123:33338) 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 -- 172.21.15.123:0/3493346702 learned_addr learned my addr 172.21.15.123:0/3493346702 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410d649700 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa4040460 0x560fa3ff7ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410e64b700 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa3ff1350 0x560fa3ff17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 -- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa4040460 msgr2=0x560fa3ff7ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa4040460 0x560fa3ff7ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 -- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa3ff1350 msgr2=0x560fa3ff17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa3ff1350 0x560fa3ff17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.264 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 -- 172.21.15.123:0/3493346702 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fa3d4b880 con 0x560fa3ff6860 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.259+0000 7f410de4a700 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa3ff6c30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40f8013040 tx=0x7f40f8086630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2a77525355d8652 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f40f2ffd700 1 -- 172.21.15.123:0/3493346702 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f805e040 con 0x560fa3ff6860 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f40f2ffd700 1 -- 172.21.15.123:0/3493346702 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f80598a0 con 0x560fa3ff6860 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f40f2ffd700 1 -- 172.21.15.123:0/3493346702 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40f80591a0 con 0x560fa3ff6860 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 -- 172.21.15.123:0/3493346702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 msgr2=0x560fa3ff6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa3ff6c30 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f40f8013040 tx=0x7f40f8086630 comp rx=0 tx=0).stop 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 -- 172.21.15.123:0/3493346702 shutdown_connections 2023-10-06T04:32:54.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa3ff6c30 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa3ff1350 0x560fa3ff17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3493346702 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa4040460 0x560fa3ff7ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 -- 172.21.15.123:0/3493346702 >> 172.21.15.123:0/3493346702 conn(0x560fa403f980 msgr2=0x560fa403fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 -- 172.21.15.123:0/3493346702 shutdown_connections 2023-10-06T04:32:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.260+0000 7f41193c2b80 1 -- 172.21.15.123:0/3493346702 wait complete. 2023-10-06T04:32:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.261+0000 7f41193c2b80 1 -- start start 2023-10-06T04:32:54.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.261+0000 7f41193c2b80 1 Processor -- start 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa403c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa401e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa3ff17c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f41193c2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa3f459d0 con 0x560fa3ff6860 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa3ef1a70 con 0x560fa4040460 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fa3ef1710 con 0x560fa3ff1350 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f41193c2b80 10 client.0 _ll_register_callbacks cb 0x560fa3d54af0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa401e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa401e1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33340/0 (socket says 172.21.15.123:33340) 2023-10-06T04:32:54.271 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 -- 172.21.15.123:0/2226620059 learned_addr learned my addr 172.21.15.123:0/2226620059 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410de4a700 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa403c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410e64b700 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa3ff17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 -- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 msgr2=0x560fa403c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa403c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 -- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 msgr2=0x560fa3ff17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa3ff17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.262+0000 7f410d649700 1 -- 172.21.15.123:0/2226620059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4108096bd0 con 0x560fa3ff6860 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f410d649700 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa401e1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4104014840 tx=0x7f4104018040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e34dd133868f8eae server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f40e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f40e7fff700 1 -- 172.21.15.123:0/2226620059 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104021630 con 0x560fa3ff6860 2023-10-06T04:32:54.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f40e7fff700 1 -- 172.21.15.123:0/2226620059 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4104021790 con 0x560fa3ff6860 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x560fa3fe9f40 con 0x560fa3ff6860 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f40e7fff700 1 -- 172.21.15.123:0/2226620059 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4104021b60 con 0x560fa3ff6860 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 10 client.5598 fetch_fsmap learned FSMap version 26 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x560fa3e38190 con 0x560fa3ff6860 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f40e7fff700 1 -- 172.21.15.123:0/2226620059 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x560fa3fe9f40 con 0x560fa3ff6860 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f40e7fff700 1 -- 172.21.15.123:0/2226620059 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 3659+0+0 (secure 0 0 0) 0x7f4104021cc0 con 0x560fa3ff6860 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 10 client.5598 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 10 client.5598 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 20 client.5598 populate_metadata read hostname 'smithi123' 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa404e960 0x560fa404ed30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 4 client.5598 mds_command: new command op to 5295 tid=0[{"prefix": "get_command_descriptions"}] 2023-10-06T04:32:54.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 --> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x560fa3fc6d10 con 0x560fa404e960 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.263+0000 7f410de4a700 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa404e960 0x560fa404ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f40e7fff700 10 client.5598 ms_handle_connect on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f410de4a700 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa404e960 0x560fa404ed30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eec704dbdd07c2a8 server_cookie=d17de1c278ed3119 in_seq=0 out_seq=0 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f40e7fff700 1 -- 172.21.15.123:0/2226620059 <== mds.0 v2:172.21.15.123:6834/571369576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+16954 (crc 0 0 0) 0x560fa3fc6d10 con 0x560fa404e960 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f40e7fff700 10 client.5598 handle_command_reply: tid=0 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 client.5598 shutdown 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 msgr2=0x560fa401e1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa401e1f0 secure :-1 s=READY pgs=507 cs=0 l=1 rev1=1 crypto rx=0x7f4104014840 tx=0x7f4104018040 comp rx=0 tx=0).stop 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 shutdown_connections 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa404e960 0x560fa404ed30 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa3ff17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.274 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f40e7fff700 0 client.5598 ms_handle_reset on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa403c870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 --2- 172.21.15.123:0/2226620059 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa401e1f0 unknown :-1 s=CLOSED pgs=507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 >> 172.21.15.123:0/2226620059 conn(0x560fa403f980 msgr2=0x560fa403d030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.264+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 shutdown_connections 2023-10-06T04:32:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.265+0000 7f41193c2b80 1 -- 172.21.15.123:0/2226620059 wait complete. 2023-10-06T04:32:54.275 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.265+0000 7f41193c2b80 20 client.5598 trim_cache size 0 max 16384 2023-10-06T04:32:54.283 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.281+0000 7f41193c2b80 1 Processor -- start 2023-10-06T04:32:54.283 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.281+0000 7f41193c2b80 1 -- start start 2023-10-06T04:32:54.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa404e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa4054d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa40552a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f41193c2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa40256b0 con 0x560fa3ff6860 2023-10-06T04:32:54.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa3ddd2d0 con 0x560fa4040460 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fa401d640 con 0x560fa3ff1350 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa4054d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa4054d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33350/0 (socket says 172.21.15.123:33350) 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 -- 172.21.15.123:0/3811698333 learned_addr learned my addr 172.21.15.123:0/3811698333 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 -- 172.21.15.123:0/3811698333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 msgr2=0x560fa404e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 --2- 172.21.15.123:0/3811698333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa404e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 -- 172.21.15.123:0/3811698333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 msgr2=0x560fa40552a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 --2- 172.21.15.123:0/3811698333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa40552a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.282+0000 7f410d649700 1 -- 172.21.15.123:0/3811698333 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x560fa403c030 con 0x560fa3ff6860 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f410d649700 1 --2- 172.21.15.123:0/3811698333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa4054d60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f410400b210 tx=0x7f4104021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebdc9b11112249fc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f40f37fe700 1 -- 172.21.15.123:0/3811698333 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104014e30 con 0x560fa3ff6860 2023-10-06T04:32:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f40f37fe700 1 -- 172.21.15.123:0/3811698333 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4104036070 con 0x560fa3ff6860 2023-10-06T04:32:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f40f37fe700 1 -- 172.21.15.123:0/3811698333 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104021630 con 0x560fa3ff6860 2023-10-06T04:32:54.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 -- 172.21.15.123:0/3811698333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 msgr2=0x560fa4054d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3811698333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa4054d60 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f410400b210 tx=0x7f4104021000 comp rx=0 tx=0).stop 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 -- 172.21.15.123:0/3811698333 shutdown_connections 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3811698333 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa40552a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3811698333 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa404e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3811698333 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa4054d60 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 -- 172.21.15.123:0/3811698333 >> 172.21.15.123:0/3811698333 conn(0x560fa403f980 msgr2=0x560fa403fd50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 -- 172.21.15.123:0/3811698333 shutdown_connections 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.283+0000 7f41193c2b80 1 -- 172.21.15.123:0/3811698333 wait complete. 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.284+0000 7f41193c2b80 1 -- start start 2023-10-06T04:32:54.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.284+0000 7f41193c2b80 1 Processor -- start 2023-10-06T04:32:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa40a0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa405ad30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f41193c2b80 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa401b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f41193c2b80 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa40a11c0 con 0x560fa3ff6860 2023-10-06T04:32:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x560fa401c5d0 con 0x560fa4040460 2023-10-06T04:32:54.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f41193c2b80 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x560fa405b7e0 con 0x560fa3ff1350 2023-10-06T04:32:54.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f41193c2b80 10 client.0 _ll_register_callbacks cb 0x560fa40193f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-10-06T04:32:54.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa405ad30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa405ad30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33362/0 (socket says 172.21.15.123:33362) 2023-10-06T04:32:54.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 -- 172.21.15.123:0/3132840958 learned_addr learned my addr 172.21.15.123:0/3132840958 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.294 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410de4a700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa40a0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410e64b700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa401b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 -- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 msgr2=0x560fa40a0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa40a0c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 -- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 msgr2=0x560fa401b400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa401b400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410d649700 1 -- 172.21.15.123:0/3132840958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x560fa3e38190 con 0x560fa3ff6860 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.285+0000 7f410e64b700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa401b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f410d649700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa405ad30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4104034040 tx=0x7f4104034070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac9635f7c10bfe43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f40e7fff700 10 client.0 ms_handle_connect on v2:172.21.15.123:3300/0 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f40e7fff700 1 -- 172.21.15.123:0/3132840958 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4104021420 con 0x560fa3ff6860 2023-10-06T04:32:54.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f40e7fff700 1 -- 172.21.15.123:0/3132840958 <== mon.0 v2:172.21.15.123:3300/0 2 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4104021580 con 0x560fa3ff6860 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x7f40f80598a0 con 0x560fa3ff6860 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f40e7fff700 1 -- 172.21.15.123:0/3132840958 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_get_version_reply(handle=1 version=26) v2 ==== 24+0+0 (secure 0 0 0) 0x7f41040293a0 con 0x560fa3ff6860 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 10 client.5604 fetch_fsmap learned FSMap version 26 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,fsmap=26,monmap=2+}) v3 -- 0x560fa3f144b0 con 0x560fa3ff6860 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f40e7fff700 1 -- 172.21.15.123:0/3132840958 <== mon.0 v2:172.21.15.123:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40f80598a0 con 0x560fa3ff6860 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f40e7fff700 1 -- 172.21.15.123:0/3132840958 <== mon.0 v2:172.21.15.123:3300/0 5 ==== fsmap(e 26) v1 ==== 3659+0+0 (secure 0 0 0) 0x7f4104029500 con 0x560fa3ff6860 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 10 client.5604 fetch_fsmap finished waiting for FSMap version 26 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 10 client.5604 resolve_mds: resolved 3:0 to role '3:0' aka daemon mds.a 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 20 client.5604 populate_metadata read hostname 'smithi123' 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa401b940 0x560fa4054d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 4 client.5604 mds_command: new command op to 5295 tid=0[{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}] 2023-10-06T04:32:54.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 --> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive", "repair"]}) v1 -- 0x560fa3ee89c0 con 0x560fa401b940 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f410de4a700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa401b940 0x560fa4054d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.286+0000 7f40e7fff700 10 client.5604 ms_handle_connect on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f410de4a700 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa401b940 0x560fa4054d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=813529ad8db424bb server_cookie=7b4df24c0786b4b1 in_seq=0 out_seq=0 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f40e7fff700 1 -- 172.21.15.123:0/3132840958 <== mds.0 v2:172.21.15.123:6834/571369576 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+110 (crc 0 0 0) 0x560fa3ee89c0 con 0x560fa401b940 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f40e7fff700 10 client.5604 handle_command_reply: tid=0 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 client.5604 shutdown 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 msgr2=0x560fa405ad30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa405ad30 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f4104034040 tx=0x7f4104034070 comp rx=0 tx=0).stop 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 shutdown_connections 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:6834/571369576,v1:172.21.15.123:6836/571369576] conn(0x560fa401b940 0x560fa4054d60 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x560fa4040460 0x560fa401b400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x560fa3ff1350 0x560fa40a0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 --2- 172.21.15.123:0/3132840958 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x560fa3ff6860 0x560fa405ad30 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 >> 172.21.15.123:0/3132840958 conn(0x560fa403f980 msgr2=0x560fa3ff71d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.287+0000 7f40e7fff700 0 client.5604 ms_handle_reset on v2:172.21.15.123:6834/571369576 2023-10-06T04:32:54.298 INFO:tasks.ceph.mds.a.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f99813f7700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-06T04:32:54.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 shutdown_connections 2023-10-06T04:32:54.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f41193c2b80 1 -- 172.21.15.123:0/3132840958 wait complete. 2023-10-06T04:32:54.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f41193c2b80 20 client.5604 trim_cache size 0 max 16384 2023-10-06T04:32:54.299 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "return_code": 0, 2023-10-06T04:32:54.299 INFO:teuthology.orchestra.run.smithi123.stdout: "scrub_tag": "dac0e9cc-05b2-4a57-9353-89c6e967f673", 2023-10-06T04:32:54.300 INFO:teuthology.orchestra.run.smithi123.stdout: "mode": "asynchronous" 2023-10-06T04:32:54.300 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:54.300 INFO:tasks.ceph.mds.a.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f99813f7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5584fbdff080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40ec066f30 msgr2=0x7f40ec0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f41100ae700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40ec066f30 0x7f40ec0693e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f4104006fd0 tx=0x7f4104008040 comp rx=0 tx=0).stop 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41080acd90 msgr2=0x7f41080cba00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.288+0000 7f41100ae700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41080acd90 0x7f41080cba00 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f410806a1a0 tx=0x7f40f801a000 comp rx=0 tx=0).stop 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 shutdown_connections 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f40ec066f30 0x7f40ec0693e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f41080acd90 0x7f41080cba00 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f41081573e0 0x7f41080ca570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 --2- 172.21.15.123:0/193582605 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f41080ae500 0x7f41080ca030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 >> 172.21.15.123:0/193582605 conn(0x7f41080cdb10 msgr2=0x7f41080b24f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 shutdown_connections 2023-10-06T04:32:54.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f41100ae700 1 -- 172.21.15.123:0/193582605 wait complete. 2023-10-06T04:32:54.302 INFO:tasks.ceph.mds.a.smithi123.stderr:2023-10-06T04:32:54.289+0000 7f99813f7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5584fbe6b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[]","memoryvalue":"(6)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T04:32:54.302 INFO:tasks.ceph.mds.a.smithi123.stderr:2023-10-06T04:32:54.293+0000 7f99813f7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 ap=5 snaprealm=0x5584fb053200 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) | dirtyscattered=0 lock=3 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5584fbdfe580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T04:32:54.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:54.420 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.418+0000 7f43451d2700 1 Processor -- start 2023-10-06T04:32:54.420 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.418+0000 7f43451d2700 1 -- start start 2023-10-06T04:32:54.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f43451d2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4340156f60 0x7f434015ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f43451d2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f434015b290 0x7f43400ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f43451d2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4340157910 con 0x7f434006cd60 2023-10-06T04:32:54.422 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f43451d2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f434006d240 con 0x7f4340156f60 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f43451d2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f434006d520 con 0x7f434015b290 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4340156f60 0x7f434015ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4340156f60 0x7f434015ad50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33378/0 (socket says 172.21.15.123:33378) 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 -- 172.21.15.123:0/3481894256 learned_addr learned my addr 172.21.15.123:0/3481894256 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433e59c700 1 --2- 172.21.15.123:0/3481894256 >> v2:172.21.15.177:3300/0 conn(0x7f434015b290 0x7f43400ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 -- 172.21.15.123:0/3481894256 >> v2:172.21.15.177:3300/0 conn(0x7f434015b290 msgr2=0x7f43400ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 --2- 172.21.15.123:0/3481894256 >> v2:172.21.15.177:3300/0 conn(0x7f434015b290 0x7f43400ad730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 -- 172.21.15.123:0/3481894256 >> v1:172.21.15.177:6789/0 conn(0x7f434006cd60 legacy=0x7f4340157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 -- 172.21.15.123:0/3481894256 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f434006d800 con 0x7f4340156f60 2023-10-06T04:32:54.423 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.419+0000 7f433ed9d700 1 --2- 172.21.15.123:0/3481894256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f434015ad50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f43340099b0 tx=0x7f4334009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48db14d4aafe1441 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f433dd9b700 1 -- 172.21.15.123:0/3481894256 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433400b7e0 con 0x7f4340156f60 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f433dd9b700 1 -- 172.21.15.123:0/3481894256 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f433400b940 con 0x7f4340156f60 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f433dd9b700 1 -- 172.21.15.123:0/3481894256 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f433400bce0 con 0x7f4340156f60 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 -- 172.21.15.123:0/3481894256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 msgr2=0x7f434015ad50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 --2- 172.21.15.123:0/3481894256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f434015ad50 secure :-1 s=READY pgs=510 cs=0 l=1 rev1=1 crypto rx=0x7f43340099b0 tx=0x7f4334009250 comp rx=0 tx=0).stop 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 -- 172.21.15.123:0/3481894256 shutdown_connections 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 --2- 172.21.15.123:0/3481894256 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f434015ad50 unknown :-1 s=CLOSED pgs=510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 --2- 172.21.15.123:0/3481894256 >> v2:172.21.15.177:3300/0 conn(0x7f434015b290 0x7f43400ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 -- 172.21.15.123:0/3481894256 >> 172.21.15.123:0/3481894256 conn(0x7f43400ce0c0 msgr2=0x7f43400ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.424 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 -- 172.21.15.123:0/3481894256 shutdown_connections 2023-10-06T04:32:54.425 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.420+0000 7f43451d2700 1 -- 172.21.15.123:0/3481894256 wait complete. 2023-10-06T04:32:54.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.421+0000 7f43451d2700 1 Processor -- start 2023-10-06T04:32:54.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.421+0000 7f43451d2700 1 -- start start 2023-10-06T04:32:54.427 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f43451d2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f434006cd60 0x7f4340154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f43451d2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f43400c2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f43451d2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f434015b290 0x7f43400c3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f43451d2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f434006a6c0 con 0x7f4340156f60 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f43451d2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43400c4220 con 0x7f434006cd60 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f43451d2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4340069ab0 con 0x7f434015b290 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f433e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f43400c2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f433e59c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f43400c2ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33384/0 (socket says 172.21.15.123:33384) 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f433e59c700 1 -- 172.21.15.123:0/4052515406 learned_addr learned my addr 172.21.15.123:0/4052515406 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.428 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f433ed9d700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f434006cd60 0x7f4340154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.422+0000 7f433f59e700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f434015b290 0x7f43400c3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f433e59c700 1 -- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f434015b290 msgr2=0x7f43400c3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f433e59c700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f434015b290 0x7f43400c3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f433e59c700 1 -- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f434006cd60 msgr2=0x7f4340154630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f433e59c700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f434006cd60 0x7f4340154630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f433e59c700 1 -- 172.21.15.123:0/4052515406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43340094f0 con 0x7f4340156f60 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f433f59e700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f434015b290 0x7f43400c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:54.429 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f433e59c700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f43400c2ed0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f433000d890 tx=0x7f4330008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f4326ffd700 1 -- 172.21.15.123:0/4052515406 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330012630 con 0x7f4340156f60 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f4326ffd700 1 -- 172.21.15.123:0/4052515406 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4330012790 con 0x7f4340156f60 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f4326ffd700 1 -- 172.21.15.123:0/4052515406 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4330012b30 con 0x7f4340156f60 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4340069d90 con 0x7f4340156f60 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.423+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f434006a350 con 0x7f4340156f60 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.424+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f434006e690 con 0x7f4340156f60 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.424+0000 7f4326ffd700 1 -- 172.21.15.123:0/4052515406 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f433001e460 con 0x7f4340156f60 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.425+0000 7f4326ffd700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f432c066f30 0x7f432c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.430 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.425+0000 7f4326ffd700 1 -- 172.21.15.123:0/4052515406 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4330084d60 con 0x7f4340156f60 2023-10-06T04:32:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.425+0000 7f433ed9d700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f432c066f30 0x7f432c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.425+0000 7f433ed9d700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f432c066f30 0x7f432c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43340099b0 tx=0x7f4334016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.431 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.429+0000 7f4326ffd700 1 -- 172.21.15.123:0/4052515406 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4330051c50 con 0x7f4340156f60 2023-10-06T04:32:54.589 INFO:tasks.ceph.mds.a.smithi123.stderr:2023-10-06T04:32:54.588+0000 7f9984bfe700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:32:54.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.651+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f434006dae0 con 0x7f4340156f60 2023-10-06T04:32:54.653 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.652+0000 7f4326ffd700 1 -- 172.21.15.123:0/4052515406 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5671 (secure 0 0 0) 0x7f4330055270 con 0x7f4340156f60 2023-10-06T04:32:54.653 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:54.654 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:32:54.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f432c066f30 msgr2=0x7f432c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f432c066f30 0x7f432c0693e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f43340099b0 tx=0x7f4334016000 comp rx=0 tx=0).stop 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 msgr2=0x7f43400c2ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f43400c2ed0 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7f433000d890 tx=0x7f4330008040 comp rx=0 tx=0).stop 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 shutdown_connections 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f432c066f30 0x7f432c0693e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f434015b290 0x7f43400c3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f434006cd60 0x7f4340154630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 --2- 172.21.15.123:0/4052515406 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4340156f60 0x7f43400c2ed0 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 >> 172.21.15.123:0/4052515406 conn(0x7f43400ce0c0 msgr2=0x7f43401552f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 shutdown_connections 2023-10-06T04:32:54.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.654+0000 7f43451d2700 1 -- 172.21.15.123:0/4052515406 wait complete. 2023-10-06T04:32:54.657 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 26 2023-10-06T04:32:54.667 INFO:tasks.cephfs.test_recovery_pool:{ "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "default_fscid": 2, "epoch": 26, "feature_flags": { "enable_multiple": true, "ever_enabled_multiple": true }, "filesystems": [ { "id": 2, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T04:32:23.822729+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 18, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [ 0 ], "flags": 19, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": false }, "fs_name": "cephfs", "in": [ 0 ], "info": {}, "last_failure": 0, "last_failure_osd_epoch": 35, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 5, "modified": "2023-10-06T04:32:33.579847+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": {} } }, { "id": 3, "mdsmap": { "balancer": "", "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "created": "2023-10-06T04:32:37.297331+0000", "damaged": [], "data_pools": [ 6 ], "enabled": true, "epoch": 26, "ever_allowed_features": 0, "explicitly_allowed_features": 0, "failed": [], "flags": 18, "flags_state": { "allow_multimds_snaps": true, "allow_snaps": true, "allow_standby_replay": false, "joinable": true }, "fs_name": "recovery_fs", "in": [ 0 ], "info": { "gid_5295": { "addr": "172.21.15.123:6836/571369576", "addrs": { "addrvec": [ { "addr": "172.21.15.123:6834", "nonce": 571369576, "type": "v2" }, { "addr": "172.21.15.123:6836", "nonce": 571369576, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5295, "incarnation": 23, "join_fscid": -1, "name": "a", "rank": 0, "state": "up:active", "state_seq": 7 } }, "last_failure": 0, "last_failure_osd_epoch": 0, "max_file_size": 1099511627776, "max_mds": 1, "metadata_pool": 7, "modified": "2023-10-06T04:32:52.264088+0000", "required_client_features": {}, "root": 0, "session_autoclose": 300, "session_timeout": 60, "standby_count_wanted": 0, "stopped": [], "tableserver": 0, "up": { "mds_0": 5295 } } } ], "standbys": [ { "addr": "172.21.15.123:6837/2896573989", "addrs": { "addrvec": [ { "addr": "172.21.15.123:6835", "nonce": 2896573989, "type": "v2" }, { "addr": "172.21.15.123:6837", "nonce": 2896573989, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 4829, "incarnation": 0, "join_fscid": -1, "name": "c", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.177:6833/1233807634", "addrs": { "addrvec": [ { "addr": "172.21.15.177:6832", "nonce": 1233807634, "type": "v2" }, { "addr": "172.21.15.177:6833", "nonce": 1233807634, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5068, "incarnation": 0, "join_fscid": -1, "name": "b", "rank": -1, "state": "up:standby", "state_seq": 1 }, { "addr": "172.21.15.177:6835/126227253", "addrs": { "addrvec": [ { "addr": "172.21.15.177:6834", "nonce": 126227253, "type": "v2" }, { "addr": "172.21.15.177:6835", "nonce": 126227253, "type": "v1" } ] }, "compat": { "compat": {}, "incompat": { "feature_1": "base v0.20", "feature_10": "snaprealm v2", "feature_2": "client writeable ranges", "feature_3": "default file layouts on dirs", "feature_4": "dir inode in separate object", "feature_5": "mds uses versioned encoding", "feature_6": "dirfrag is stored in omap", "feature_7": "mds uses inline data", "feature_8": "no anchor table", "feature_9": "file layout v2" }, "ro_compat": {} }, "epoch": 19, "export_targets": [], "features": 4540138320759226367, "flags": 0, "gid": 5071, "incarnation": 0, "join_fscid": -1, "name": "d", "rank": -1, "state": "up:standby", "state_seq": 1 } ] } 2023-10-06T04:32:54.668 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-10-06T04:32:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.784+0000 7fa43ed67700 1 Processor -- start 2023-10-06T04:32:54.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.784+0000 7fa43ed67700 1 -- start start 2023-10-06T04:32:54.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa43ed67700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa43806e960 0x7fa4380c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa43ed67700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380c8580 con 0x7fa4381560c0 2023-10-06T04:32:54.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa43ed67700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4380c8860 con 0x7fa43806f080 2023-10-06T04:32:54.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa43ed67700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4380c8b40 con 0x7fa43806e960 2023-10-06T04:32:54.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa437fff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa43806e960 0x7fa4380c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa43cb03700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa4381560c0 0x7fa43806e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36676/0 (socket says 172.21.15.123:36676) 2023-10-06T04:32:54.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa43cb03700 1 -- 172.21.15.123:0/184085785 learned_addr learned my addr 172.21.15.123:0/184085785 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:54.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa437fff700 1 -- 172.21.15.123:0/184085785 >> v1:172.21.15.177:6789/0 conn(0x7fa43806f080 legacy=0x7fa4380ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.795 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa437fff700 1 -- 172.21.15.123:0/184085785 >> v1:172.21.15.123:6789/0 conn(0x7fa4381560c0 legacy=0x7fa43806e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa437fff700 1 -- 172.21.15.123:0/184085785 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4380c8e20 con 0x7fa43806e960 2023-10-06T04:32:54.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa4377fe700 1 -- 172.21.15.123:0/184085785 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1410992655 0 0) 0x7fa4380c8580 con 0x7fa4381560c0 2023-10-06T04:32:54.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.785+0000 7fa437fff700 1 --2- 172.21.15.123:0/184085785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43806e960 0x7fa4380c7fa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa4240098e0 tx=0x7fa4240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1f8aabc5142e5cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa4377fe700 1 -- 172.21.15.123:0/184085785 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa424009e80 con 0x7fa43806e960 2023-10-06T04:32:54.796 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa4377fe700 1 -- 172.21.15.123:0/184085785 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa42400b860 con 0x7fa43806e960 2023-10-06T04:32:54.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa4377fe700 1 -- 172.21.15.123:0/184085785 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa42400bbe0 con 0x7fa43806e960 2023-10-06T04:32:54.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa43ed67700 1 -- 172.21.15.123:0/184085785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43806e960 msgr2=0x7fa4380c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa43ed67700 1 --2- 172.21.15.123:0/184085785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43806e960 0x7fa4380c7fa0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fa4240098e0 tx=0x7fa4240091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa43ed67700 1 -- 172.21.15.123:0/184085785 shutdown_connections 2023-10-06T04:32:54.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa43ed67700 1 --2- 172.21.15.123:0/184085785 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa43806e960 0x7fa4380c7fa0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.798 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa43ed67700 1 -- 172.21.15.123:0/184085785 >> 172.21.15.123:0/184085785 conn(0x7fa4380cda80 msgr2=0x7fa4380c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:54.798 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa43ed67700 1 -- 172.21.15.123:0/184085785 shutdown_connections 2023-10-06T04:32:54.798 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.786+0000 7fa43ed67700 1 -- 172.21.15.123:0/184085785 wait complete. 2023-10-06T04:32:54.798 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.787+0000 7fa43ed67700 1 Processor -- start 2023-10-06T04:32:54.798 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43ed67700 1 -- start start 2023-10-06T04:32:54.798 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43ed67700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa43806e960 0x7fa4380ab0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.798 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43ed67700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa43806f080 0x7fa43815b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43ed67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4381560c0 0x7fa438067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43ed67700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa438067700 con 0x7fa4381560c0 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43ed67700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380ab680 con 0x7fa43806e960 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43ed67700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4380ab960 con 0x7fa43806f080 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43d304700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4381560c0 0x7fa438067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43d304700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4381560c0 0x7fa438067e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33388/0 (socket says 172.21.15.123:33388) 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43d304700 1 -- 172.21.15.123:0/2332173074 learned_addr learned my addr 172.21.15.123:0/2332173074 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa437fff700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa43806f080 0x7fa43815b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.788+0000 7fa43cb03700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa43806e960 0x7fa4380ab0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43d304700 1 -- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa43806f080 msgr2=0x7fa43815b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43d304700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa43806f080 0x7fa43815b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43d304700 1 -- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa43806e960 msgr2=0x7fa4380ab0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43d304700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa43806e960 0x7fa4380ab0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43d304700 1 -- 172.21.15.123:0/2332173074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa424009420 con 0x7fa4381560c0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43d304700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4381560c0 0x7fa438067e90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c8e0 tx=0x7fa430001710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa434ff9700 1 -- 172.21.15.123:0/2332173074 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430012630 con 0x7fa4381560c0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa434ff9700 1 -- 172.21.15.123:0/2332173074 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa430012790 con 0x7fa4381560c0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa438067970 con 0x7fa4381560c0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa434ff9700 1 -- 172.21.15.123:0/2332173074 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa430012b50 con 0x7fa4381560c0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.789+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa43815bc20 con 0x7fa4381560c0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.790+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa438067b60 con 0x7fa4381560c0 2023-10-06T04:32:54.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.790+0000 7fa434ff9700 1 -- 172.21.15.123:0/2332173074 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa43001a3a0 con 0x7fa4381560c0 2023-10-06T04:32:54.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.791+0000 7fa434ff9700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa41c066f30 0x7fa41c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:54.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.791+0000 7fa434ff9700 1 -- 172.21.15.123:0/2332173074 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa430084890 con 0x7fa4381560c0 2023-10-06T04:32:54.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.791+0000 7fa43cb03700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa41c066f30 0x7fa41c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:54.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.791+0000 7fa43cb03700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa41c066f30 0x7fa41c0693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa42c000c00 tx=0x7fa42c007f30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:54.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:54.795+0000 7fa434ff9700 1 -- 172.21.15.123:0/2332173074 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa430055210 con 0x7fa4381560c0 2023-10-06T04:32:55.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.020+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa43815bc20 con 0x7fa4381560c0 2023-10-06T04:32:55.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.021+0000 7fa434ff9700 1 -- 172.21.15.123:0/2332173074 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v26) v1 ==== 53+0+156 (secure 0 0 0) 0x7fa43815bc20 con 0x7fa4381560c0 2023-10-06T04:32:55.022 INFO:teuthology.orchestra.run.smithi123.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-10-06T04:32:55.022 INFO:teuthology.orchestra.run.smithi123.stdout:name: recovery_fs, metadata pool: recovery_fs_metadata, data pools: [cephfs_data ] 2023-10-06T04:32:55.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa41c066f30 msgr2=0x7fa41c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:55.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa41c066f30 0x7fa41c0693e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fa42c000c00 tx=0x7fa42c007f30 comp rx=0 tx=0).stop 2023-10-06T04:32:55.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4381560c0 msgr2=0x7fa438067e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:55.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4381560c0 0x7fa438067e90 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7fa43000c8e0 tx=0x7fa430001710 comp rx=0 tx=0).stop 2023-10-06T04:32:55.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 shutdown_connections 2023-10-06T04:32:55.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa41c066f30 0x7fa41c0693e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:55.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa4381560c0 0x7fa438067e90 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:55.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa43806e960 0x7fa4380ab0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:55.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.022+0000 7fa43ed67700 1 --2- 172.21.15.123:0/2332173074 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa43806f080 0x7fa43815b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:55.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.023+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 >> 172.21.15.123:0/2332173074 conn(0x7fa4380cda80 msgr2=0x7fa4381554d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:55.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.023+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 shutdown_connections 2023-10-06T04:32:55.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:55.023+0000 7fa43ed67700 1 -- 172.21.15.123:0/2332173074 wait complete. 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:Mounting Ceph FS recovery_fs; just confirmed its presence on cluster 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi123.front.sepia.ceph.com 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:self.cephfs_name = recovery_fs 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-10-06T04:32:55.036 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:32:55.037 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:55.037 DEBUG:teuthology.orchestra.run.smithi123:> ip addr 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: inet 127.0.0.1/8 scope host lo 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 ::1/128 scope host 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6c brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:55.056 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f0 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:d8:da:6d brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: altname enp7s0f1 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f0 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 0c:c4:7a:8f:d3:f1 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: inet 172.21.15.123/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft 42440sec preferred_lft 42440sec 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::ec4:7aff:fe8f:d3f1/64 scope link 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:55.057 INFO:teuthology.orchestra.run.smithi123.stdout:6: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:ca:fa:1b:04:98 brd ff:ff:ff:ff:ff:ff 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::5073:dcff:fe45:d49e/64 scope link 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout:7: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:e0:39:ac:6a:27 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::44e0:39ff:feac:6a27/64 scope link 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout:8: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: link/ether 46:ca:fa:1b:04:98 brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:32:55.058 INFO:teuthology.orchestra.run.smithi123.stdout: inet6 fe80::44ca:faff:fe1b:498/64 scope link 2023-10-06T04:32:55.059 INFO:teuthology.orchestra.run.smithi123.stdout: valid_lft forever preferred_lft forever 2023-10-06T04:32:55.059 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:55.059 DEBUG:teuthology.orchestra.run.smithi123:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-10-06T04:32:55.122 INFO:teuthology.orchestra.run.smithi123.stdout:1 2023-10-06T04:32:55.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:55.123 DEBUG:teuthology.orchestra.run.smithi123:> route 2023-10-06T04:32:55.183 INFO:teuthology.orchestra.run.smithi123.stdout:Kernel IP routing table 2023-10-06T04:32:55.183 INFO:teuthology.orchestra.run.smithi123.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-10-06T04:32:55.183 INFO:teuthology.orchestra.run.smithi123.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2023-10-06T04:32:55.183 INFO:teuthology.orchestra.run.smithi123.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2023-10-06T04:32:55.183 INFO:teuthology.orchestra.run.smithi123.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-10-06T04:32:55.184 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:55.184 DEBUG:teuthology.orchestra.run.smithi123:> (cd / && exec bash -c ' 2023-10-06T04:32:55.184 DEBUG:teuthology.orchestra.run.smithi123:> set -e 2023-10-06T04:32:55.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2023-10-06T04:32:55.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2023-10-06T04:32:55.184 DEBUG:teuthology.orchestra.run.smithi123:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2023-10-06T04:32:55.184 DEBUG:teuthology.orchestra.run.smithi123:> ') 2023-10-06T04:32:55.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:55.289 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-10-06T04:32:55.306 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:32:55.307 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:32:55.307 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:32:55.307 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:55.307 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link set brx.0 up 2023-10-06T04:32:55.375 INFO:tasks.cephfs.fuse_mount:Client client.0 config is {} 2023-10-06T04:32:55.375 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:32:55.375 DEBUG:teuthology.orchestra.run.smithi123:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:55.433 INFO:teuthology.orchestra.run.smithi123.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-10-06T04:32:55.433 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-10-06T04:32:55.433 DEBUG:teuthology.orchestra.run.smithi123:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:55.491 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:32:55.560 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:32:55.560 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:32:55.626 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:32:55.627 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:32:55.628 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:55.628 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:32:55.686 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2023-10-06T04:32:55.686 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.0 --id 0 --client_fs=recovery_fs) 2023-10-06T04:32:55.729 DEBUG:teuthology.orchestra.run.smithi123:> sudo modprobe fuse 2023-10-06T04:32:55.755 INFO:teuthology.orchestra.run:Running command with timeout 30 2023-10-06T04:32:55.756 DEBUG:teuthology.orchestra.run.smithi123:> sudo mount -t fusectl /sys/fs/fuse/connections /sys/fs/fuse/connections 2023-10-06T04:32:55.811 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:2023-10-06T04:32:55.810+0000 7f07f9393540 -1 init, newargv = 0x5641262ef840 newargc=15 2023-10-06T04:32:55.812 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[137465]: starting ceph client 2023-10-06T04:32:55.816 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[137465]: starting fuse 2023-10-06T04:32:55.826 INFO:teuthology.orchestra.run.smithi123.stderr:mount: /sys/fs/fuse/connections: /sys/fs/fuse/connections already mounted or mount point busy. 2023-10-06T04:32:55.827 DEBUG:teuthology.orchestra.run:got remote process result: 32 2023-10-06T04:32:55.827 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:55.828 DEBUG:teuthology.orchestra.run.smithi123:> ls /sys/fs/fuse/connections 2023-10-06T04:32:55.886 INFO:teuthology.orchestra.run.smithi123.stdout:46 2023-10-06T04:32:55.886 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [46] 2023-10-06T04:32:55.887 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:32:55.887 DEBUG:teuthology.orchestra.run.smithi123:> import glob 2023-10-06T04:32:55.887 DEBUG:teuthology.orchestra.run.smithi123:> import re 2023-10-06T04:32:55.887 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-10-06T04:32:55.887 DEBUG:teuthology.orchestra.run.smithi123:> import subprocess 2023-10-06T04:32:55.888 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:55.888 DEBUG:teuthology.orchestra.run.smithi123:> def _find_admin_socket(client_name): 2023-10-06T04:32:55.888 DEBUG:teuthology.orchestra.run.smithi123:> asok_path = "/var/run/ceph/ceph-client.0.*.asok" 2023-10-06T04:32:55.888 DEBUG:teuthology.orchestra.run.smithi123:> files = glob.glob(asok_path) 2023-10-06T04:32:55.888 DEBUG:teuthology.orchestra.run.smithi123:> mountpoint = "/home/ubuntu/cephtest/mnt.0" 2023-10-06T04:32:55.888 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:55.889 DEBUG:teuthology.orchestra.run.smithi123:> # Given a non-glob path, it better be there 2023-10-06T04:32:55.889 DEBUG:teuthology.orchestra.run.smithi123:> if "*" not in asok_path: 2023-10-06T04:32:55.889 DEBUG:teuthology.orchestra.run.smithi123:> assert(len(files) == 1) 2023-10-06T04:32:55.889 DEBUG:teuthology.orchestra.run.smithi123:> return files[0] 2023-10-06T04:32:55.890 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:55.890 DEBUG:teuthology.orchestra.run.smithi123:> for f in files: 2023-10-06T04:32:55.890 DEBUG:teuthology.orchestra.run.smithi123:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2023-10-06T04:32:55.890 DEBUG:teuthology.orchestra.run.smithi123:> if os.path.exists("/proc/{0}".format(pid)): 2023-10-06T04:32:55.890 DEBUG:teuthology.orchestra.run.smithi123:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2023-10-06T04:32:55.890 DEBUG:teuthology.orchestra.run.smithi123:> contents = proc_f.read() 2023-10-06T04:32:55.891 DEBUG:teuthology.orchestra.run.smithi123:> if mountpoint in contents: 2023-10-06T04:32:55.891 DEBUG:teuthology.orchestra.run.smithi123:> return f 2023-10-06T04:32:55.891 DEBUG:teuthology.orchestra.run.smithi123:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2023-10-06T04:32:55.891 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:55.891 DEBUG:teuthology.orchestra.run.smithi123:> print(_find_admin_socket("client.0")) 2023-10-06T04:32:55.891 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-10-06T04:32:56.009 INFO:teuthology.orchestra.run.smithi123.stdout:/var/run/ceph/ceph-client.0.137465.asok 2023-10-06T04:32:56.191 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.0.137465.asok 2023-10-06T04:32:56.192 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:56.192 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.0.137465.asok status 2023-10-06T04:32:56.347 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:56.347 INFO:teuthology.orchestra.run.smithi123.stdout: "metadata": { 2023-10-06T04:32:56.347 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_sha1": "736ac5ba72cb47af7d6f6666b1e100f7e20a20e2", 2023-10-06T04:32:56.347 INFO:teuthology.orchestra.run.smithi123.stdout: "ceph_version": "ceph version 17.2.6-1296-g736ac5ba (736ac5ba72cb47af7d6f6666b1e100f7e20a20e2) quincy (stable)", 2023-10-06T04:32:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: "entity_id": "0", 2023-10-06T04:32:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: "hostname": "smithi123", 2023-10-06T04:32:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.0", 2023-10-06T04:32:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: "pid": "137465", 2023-10-06T04:32:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: "root": "/" 2023-10-06T04:32:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:56.349 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_count": 0, 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "dentry_pinned_count": 0, 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "id": 5628, 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "inst": { 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "name": { 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "client", 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "num": 5628 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:32:56.350 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 854662119 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: } 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": { 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "type": "v1", 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "addr": "192.168.0.1:0", 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "nonce": 854662119 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: }, 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "inst_str": "client.5628 192.168.0.1:0/854662119", 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "addr_str": "192.168.0.1:0/854662119", 2023-10-06T04:32:56.351 INFO:teuthology.orchestra.run.smithi123.stdout: "inode_count": 1, 2023-10-06T04:32:56.352 INFO:teuthology.orchestra.run.smithi123.stdout: "mds_epoch": 26, 2023-10-06T04:32:56.352 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch": 42, 2023-10-06T04:32:56.352 INFO:teuthology.orchestra.run.smithi123.stdout: "osd_epoch_barrier": 0, 2023-10-06T04:32:56.352 INFO:teuthology.orchestra.run.smithi123.stdout: "blocklisted": false, 2023-10-06T04:32:56.352 INFO:teuthology.orchestra.run.smithi123.stdout: "fs_name": "recovery_fs" 2023-10-06T04:32:56.352 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:56.357 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:56.357 DEBUG:teuthology.orchestra.run.smithi123:> stat --file-system '--printf=%T 2023-10-06T04:32:56.357 DEBUG:teuthology.orchestra.run.smithi123:> ' -- /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:56.376 INFO:teuthology.orchestra.run.smithi123.stdout:fuseblk 2023-10-06T04:32:56.376 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:56.376 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:56.377 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:56.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2023-10-06T04:32:56.444 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls subdir) 2023-10-06T04:32:56.506 INFO:teuthology.orchestra.run.smithi123.stdout:sixmegs 2023-10-06T04:32:56.506 DEBUG:teuthology.orchestra.run.smithi123:> adjust-ulimits daemon-helper kill python3 -c ' 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> import os 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> import stat 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> import json 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> import sys 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> try: 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> s = os.stat('"'"'/home/ubuntu/cephtest/mnt.0/subdir/sixmegs'"'"') 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> except OSError as e: 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> sys.exit(e.errno) 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> 2023-10-06T04:32:56.507 DEBUG:teuthology.orchestra.run.smithi123:> attrs = ["st_mode", "st_ino", "st_dev", "st_nlink", "st_uid", "st_gid", "st_size", "st_atime", "st_mtime", "st_ctime"] 2023-10-06T04:32:56.508 DEBUG:teuthology.orchestra.run.smithi123:> print(json.dumps( 2023-10-06T04:32:56.508 DEBUG:teuthology.orchestra.run.smithi123:> dict([(a, getattr(s, a)) for a in attrs]), 2023-10-06T04:32:56.508 DEBUG:teuthology.orchestra.run.smithi123:> indent=2)) 2023-10-06T04:32:56.508 DEBUG:teuthology.orchestra.run.smithi123:> ' 2023-10-06T04:32:56.618 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:32:56.619 INFO:teuthology.orchestra.run.smithi123.stdout: "st_mode": 33088, 2023-10-06T04:32:56.619 INFO:teuthology.orchestra.run.smithi123.stdout: "st_ino": 1099511627777, 2023-10-06T04:32:56.619 INFO:teuthology.orchestra.run.smithi123.stdout: "st_dev": 46, 2023-10-06T04:32:56.620 INFO:teuthology.orchestra.run.smithi123.stdout: "st_nlink": 1, 2023-10-06T04:32:56.620 INFO:teuthology.orchestra.run.smithi123.stdout: "st_uid": 0, 2023-10-06T04:32:56.620 INFO:teuthology.orchestra.run.smithi123.stdout: "st_gid": 0, 2023-10-06T04:32:56.620 INFO:teuthology.orchestra.run.smithi123.stdout: "st_size": 6291456, 2023-10-06T04:32:56.621 INFO:teuthology.orchestra.run.smithi123.stdout: "st_atime": 1696566753.0, 2023-10-06T04:32:56.621 INFO:teuthology.orchestra.run.smithi123.stdout: "st_mtime": 1696566753.0, 2023-10-06T04:32:56.621 INFO:teuthology.orchestra.run.smithi123.stdout: "st_ctime": 1696566753.0 2023-10-06T04:32:56.621 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:32:56.803 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-10-06T04:32:56.803 DEBUG:teuthology.orchestra.run.smithi123:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:32:56.884 DEBUG:teuthology.orchestra.run.smithi177:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-10-06T04:32:56.934 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-10-06T04:32:56.934 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi123.front.sepia.ceph.com... 2023-10-06T04:32:56.934 INFO:teuthology.orchestra.run:Running command with timeout 1800 2023-10-06T04:32:56.934 DEBUG:teuthology.orchestra.run.smithi123:> sudo fusermount -u /home/ubuntu/cephtest/mnt.0 2023-10-06T04:32:56.975 INFO:tasks.cephfs.fuse_mount.ceph-fuse.0.smithi123.stderr:ceph-fuse[137465]: fuse finished with error 0 and tester_r 0 2023-10-06T04:32:56.976 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:32:56.976 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:56.977 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-10-06T04:32:57.062 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:32:57.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:57.063 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-10-06T04:32:57.084 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:32:57.084 INFO:teuthology.orchestra.run.smithi177.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.1': No such file or directory 2023-10-06T04:32:57.085 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:32:57.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:57.085 DEBUG:teuthology.orchestra.run.smithi123:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-10-06T04:32:57.120 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:32:57.121 INFO:teuthology.orchestra.run.smithi123.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.2': No such file or directory 2023-10-06T04:32:57.121 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:32:57.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:32:57.122 DEBUG:teuthology.orchestra.run.smithi177:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-10-06T04:32:57.145 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:32:57.146 INFO:teuthology.orchestra.run.smithi177.stderr:rmdir: failed to remove '/home/ubuntu/cephtest/mnt.3': No such file or directory 2023-10-06T04:32:57.146 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.277+0000 7f2911df5700 1 Processor -- start 2023-10-06T04:32:57.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.277+0000 7f2911df5700 1 -- start start 2023-10-06T04:32:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f2911df5700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f290c06a540 0x7f290c15d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f2911df5700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c0690a0 con 0x7f290c06bb60 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f2911df5700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c15dd40 con 0x7f290c13f600 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f2911df5700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290c068770 con 0x7f290c06a540 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290bfff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f290c06a540 0x7f290c15d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290affd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f290c06bb60 0x7f290c15d010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36682/0 (socket says 172.21.15.123:36682) 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290affd700 1 -- 172.21.15.123:0/2702961489 learned_addr learned my addr 172.21.15.123:0/2702961489 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290bfff700 1 -- 172.21.15.123:0/2702961489 >> v1:172.21.15.177:6789/0 conn(0x7f290c13f600 legacy=0x7f290c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290bfff700 1 -- 172.21.15.123:0/2702961489 >> v1:172.21.15.123:6789/0 conn(0x7f290c06bb60 legacy=0x7f290c15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290bfff700 1 -- 172.21.15.123:0/2702961489 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c068a50 con 0x7f290c06a540 2023-10-06T04:32:57.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290bfff700 1 --2- 172.21.15.123:0/2702961489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c06a540 0x7f290c15d720 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2904009e60 tx=0x7f2904009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=266d8fea2c51025b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290a7fc700 1 -- 172.21.15.123:0/2702961489 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f290400e070 con 0x7f290c06a540 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290a7fc700 1 -- 172.21.15.123:0/2702961489 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f290400bbf0 con 0x7f290c06a540 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.278+0000 7f290a7fc700 1 -- 172.21.15.123:0/2702961489 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2904012430 con 0x7f290c06a540 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.279+0000 7f2911df5700 1 -- 172.21.15.123:0/2702961489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c06a540 msgr2=0x7f290c15d720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.279+0000 7f2911df5700 1 --2- 172.21.15.123:0/2702961489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c06a540 0x7f290c15d720 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7f2904009e60 tx=0x7f2904009780 comp rx=0 tx=0).stop 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.279+0000 7f2911df5700 1 -- 172.21.15.123:0/2702961489 shutdown_connections 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.279+0000 7f2911df5700 1 --2- 172.21.15.123:0/2702961489 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c06a540 0x7f290c15d720 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.279+0000 7f2911df5700 1 -- 172.21.15.123:0/2702961489 >> 172.21.15.123:0/2702961489 conn(0x7f290c0ce0c0 msgr2=0x7f290c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:57.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.279+0000 7f2911df5700 1 -- 172.21.15.123:0/2702961489 shutdown_connections 2023-10-06T04:32:57.283 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.279+0000 7f2911df5700 1 -- 172.21.15.123:0/2702961489 wait complete. 2023-10-06T04:32:57.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.281+0000 7f2911df5700 1 Processor -- start 2023-10-06T04:32:57.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.281+0000 7f2911df5700 1 -- start start 2023-10-06T04:32:57.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f2911df5700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f290c06bb60 0x7f290c14bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f2911df5700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c13f600 0x7f290c06d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f2911df5700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f290c149180 0x7f290c06d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f2911df5700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0c6c60 con 0x7f290c13f600 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f2911df5700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0c6330 con 0x7f290c06bb60 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f2911df5700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0c6610 con 0x7f290c149180 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c13f600 0x7f290c06d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c13f600 0x7f290c06d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33410/0 (socket says 172.21.15.123:33410) 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 -- 172.21.15.123:0/1046628780 learned_addr learned my addr 172.21.15.123:0/1046628780 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290b7fe700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f290c06bb60 0x7f290c14bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290bfff700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f290c149180 0x7f290c06d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 -- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f290c149180 msgr2=0x7f290c06d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f290c149180 0x7f290c06d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 -- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f290c06bb60 msgr2=0x7f290c14bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f290c06bb60 0x7f290c14bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 -- 172.21.15.123:0/1046628780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904009a20 con 0x7f290c13f600 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290b7fe700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f290c06bb60 0x7f290c14bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f290affd700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c13f600 0x7f290c06d420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00c7c0 tx=0x7f28fc007f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f28f37fe700 1 -- 172.21.15.123:0/1046628780 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc005b30 con 0x7f290c13f600 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f28f37fe700 1 -- 172.21.15.123:0/1046628780 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc005c90 con 0x7f290c13f600 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.282+0000 7f28f37fe700 1 -- 172.21.15.123:0/1046628780 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc004a80 con 0x7f290c13f600 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.283+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c0c68f0 con 0x7f290c13f600 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.283+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c159bb0 con 0x7f290c13f600 2023-10-06T04:32:57.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.283+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c06e690 con 0x7f290c13f600 2023-10-06T04:32:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.284+0000 7f28f37fe700 1 -- 172.21.15.123:0/1046628780 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f28fc0070d0 con 0x7f290c13f600 2023-10-06T04:32:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.284+0000 7f28f37fe700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f28f8066f30 0x7f28f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.284+0000 7f290b7fe700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f28f8066f30 0x7f28f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.284+0000 7f28f37fe700 1 -- 172.21.15.123:0/1046628780 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f28fc083c80 con 0x7f290c13f600 2023-10-06T04:32:57.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.285+0000 7f290b7fe700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f28f8066f30 0x7f28f80693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2900009a80 tx=0x7f2900008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:57.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.289+0000 7f28f37fe700 1 -- 172.21.15.123:0/1046628780 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f28fc04b5a0 con 0x7f290c13f600 2023-10-06T04:32:57.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.511+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f290c068d30 con 0x7f290c13f600 2023-10-06T04:32:57.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.512+0000 7f28f37fe700 1 -- 172.21.15.123:0/1046628780 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5671 (secure 0 0 0) 0x7f28fc050b70 con 0x7f290c13f600 2023-10-06T04:32:57.514 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:57.514 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f28f8066f30 msgr2=0x7f28f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f28f8066f30 0x7f28f80693e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f2900009a80 tx=0x7f2900008040 comp rx=0 tx=0).stop 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c13f600 msgr2=0x7f290c06d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c13f600 0x7f290c06d420 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00c7c0 tx=0x7f28fc007f70 comp rx=0 tx=0).stop 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 shutdown_connections 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f28f8066f30 0x7f28f80693e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f290c149180 0x7f290c06d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f290c06bb60 0x7f290c14bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 --2- 172.21.15.123:0/1046628780 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f290c13f600 0x7f290c06d420 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.516 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 >> 172.21.15.123:0/1046628780 conn(0x7f290c0ce0c0 msgr2=0x7f290c157450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 shutdown_connections 2023-10-06T04:32:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.514+0000 7f2911df5700 1 -- 172.21.15.123:0/1046628780 wait complete. 2023-10-06T04:32:57.517 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 26 2023-10-06T04:32:57.527 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:57.646 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 Processor -- start 2023-10-06T04:32:57.646 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 -- start start 2023-10-06T04:32:57.648 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd0980688c0 0x7fd09806a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd09806a930 0x7fd098156de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd098068c90 0x7fd098157320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0980cc620 con 0x7fd0980688c0 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0980cc190 con 0x7fd098068c90 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.645+0000 7fd09ddae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd098157860 con 0x7fd09806a930 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd0980688c0 0x7fd09806a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd0980688c0 0x7fd09806a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33422/0 (socket says 172.21.15.123:33422) 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 -- 172.21.15.123:0/4174256716 learned_addr learned my addr 172.21.15.123:0/4174256716 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd096ffd700 1 --2- 172.21.15.123:0/4174256716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd09806a930 0x7fd098156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd097fff700 1 --2- 172.21.15.123:0/4174256716 >> v2:172.21.15.177:3300/0 conn(0x7fd098068c90 0x7fd098157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.649 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 -- 172.21.15.123:0/4174256716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd09806a930 msgr2=0x7fd098156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 --2- 172.21.15.123:0/4174256716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd09806a930 0x7fd098156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 -- 172.21.15.123:0/4174256716 >> v2:172.21.15.177:3300/0 conn(0x7fd098068c90 msgr2=0x7fd098157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 --2- 172.21.15.123:0/4174256716 >> v2:172.21.15.177:3300/0 conn(0x7fd098068c90 0x7fd098157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 -- 172.21.15.123:0/4174256716 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0980cb580 con 0x7fd0980688c0 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0977fe700 1 --2- 172.21.15.123:0/4174256716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09806a3f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd0980695e0 tx=0x7fd088009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181100912834f8e6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0967fc700 1 -- 172.21.15.123:0/4174256716 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08800b860 con 0x7fd0980688c0 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0967fc700 1 -- 172.21.15.123:0/4174256716 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd08800b9c0 con 0x7fd0980688c0 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.646+0000 7fd0967fc700 1 -- 172.21.15.123:0/4174256716 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd08800bd60 con 0x7fd0980688c0 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 -- 172.21.15.123:0/4174256716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 msgr2=0x7fd09806a3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 --2- 172.21.15.123:0/4174256716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09806a3f0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7fd0980695e0 tx=0x7fd088009250 comp rx=0 tx=0).stop 2023-10-06T04:32:57.650 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 -- 172.21.15.123:0/4174256716 shutdown_connections 2023-10-06T04:32:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 --2- 172.21.15.123:0/4174256716 >> v2:172.21.15.177:3300/0 conn(0x7fd098068c90 0x7fd098157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 --2- 172.21.15.123:0/4174256716 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09806a3f0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 --2- 172.21.15.123:0/4174256716 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd09806a930 0x7fd098156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 -- 172.21.15.123:0/4174256716 >> 172.21.15.123:0/4174256716 conn(0x7fd0980c9c90 msgr2=0x7fd09806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 -- 172.21.15.123:0/4174256716 shutdown_connections 2023-10-06T04:32:57.651 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.647+0000 7fd09ddae700 1 -- 172.21.15.123:0/4174256716 wait complete. 2023-10-06T04:32:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.648+0000 7fd09ddae700 1 Processor -- start 2023-10-06T04:32:57.654 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.648+0000 7fd09ddae700 1 -- start start 2023-10-06T04:32:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd09ddae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09816a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd09ddae700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd098068c90 0x7fd0980b0ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd09ddae700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd09806a930 0x7fd0980b1410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd09ddae700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980c7dc0 con 0x7fd0980688c0 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd09ddae700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd09816ab80 con 0x7fd09806a930 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd09ddae700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0980c71b0 con 0x7fd098068c90 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09816a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09816a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33436/0 (socket says 172.21.15.123:33436) 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 -- 172.21.15.123:0/1087276918 learned_addr learned my addr 172.21.15.123:0/1087276918 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd097fff700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd09806a930 0x7fd0980b1410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd096ffd700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd098068c90 0x7fd0980b0ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 -- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd098068c90 msgr2=0x7fd0980b0ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.656 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd098068c90 0x7fd0980b0ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 -- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd09806a930 msgr2=0x7fd0980b1410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd09806a930 0x7fd0980b1410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 -- 172.21.15.123:0/1087276918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0880094f0 con 0x7fd0980688c0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd096ffd700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd098068c90 0x7fd0980b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd097fff700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd09806a930 0x7fd0980b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd0977fe700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09816a640 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd0980c8fe0 tx=0x7fd088017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd07f7fe700 1 -- 172.21.15.123:0/1087276918 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088024440 con 0x7fd0980688c0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd07f7fe700 1 -- 172.21.15.123:0/1087276918 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0880245a0 con 0x7fd0980688c0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd07f7fe700 1 -- 172.21.15.123:0/1087276918 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd088024960 con 0x7fd0980688c0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.649+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0980c7490 con 0x7fd0980688c0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.650+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0980c7a50 con 0x7fd0980688c0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.650+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd09806cc70 con 0x7fd0980688c0 2023-10-06T04:32:57.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.651+0000 7fd07f7fe700 1 -- 172.21.15.123:0/1087276918 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd08801a420 con 0x7fd0980688c0 2023-10-06T04:32:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.651+0000 7fd07f7fe700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd084066f60 0x7fd084069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.651+0000 7fd07f7fe700 1 -- 172.21.15.123:0/1087276918 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd088086600 con 0x7fd0980688c0 2023-10-06T04:32:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.651+0000 7fd096ffd700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd084066f60 0x7fd084069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.652+0000 7fd096ffd700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd084066f60 0x7fd084069410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009990 tx=0x7fd08c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:57.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.655+0000 7fd07f7fe700 1 -- 172.21.15.123:0/1087276918 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd0880534f0 con 0x7fd0980688c0 2023-10-06T04:32:57.838 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.837+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd0980cb860 con 0x7fd0980688c0 2023-10-06T04:32:57.839 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.838+0000 7fd07f7fe700 1 -- 172.21.15.123:0/1087276918 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18770 (secure 0 0 0) 0x7fd088056b10 con 0x7fd0980688c0 2023-10-06T04:32:57.840 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:57.840 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:38.586940+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:32:57.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd084066f60 msgr2=0x7fd084069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd084066f60 0x7fd084069410 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fd08c009990 tx=0x7fd08c008040 comp rx=0 tx=0).stop 2023-10-06T04:32:57.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 msgr2=0x7fd09816a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09816a640 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fd0980c8fe0 tx=0x7fd088017000 comp rx=0 tx=0).stop 2023-10-06T04:32:57.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 shutdown_connections 2023-10-06T04:32:57.842 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd084066f60 0x7fd084069410 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd0980688c0 0x7fd09816a640 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd09806a930 0x7fd0980b1410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 --2- 172.21.15.123:0/1087276918 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd098068c90 0x7fd0980b0ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 >> 172.21.15.123:0/1087276918 conn(0x7fd0980c9c90 msgr2=0x7fd0980c1dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:57.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 shutdown_connections 2023-10-06T04:32:57.843 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.840+0000 7fd09ddae700 1 -- 172.21.15.123:0/1087276918 wait complete. 2023-10-06T04:32:57.854 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 5, 'pool_name': 'cephfs_metadata', 'create_time': '2023-10-06T04:32:21.900854+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '33', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:32:22.782310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'recovery_fs'}}}, {'pool': 7, 'pool_name': 'recovery_fs_metadata', 'create_time': '2023-10-06T04:32:35.940980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'recovery_fs'}}}] 2023-10-06T04:32:57.854 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:32:57.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.974+0000 7f0b8321d700 1 Processor -- start 2023-10-06T04:32:57.976 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.974+0000 7f0b8321d700 1 -- start start 2023-10-06T04:32:57.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b8321d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c1597d0 0x7f0b7c068770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b8321d700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b7c0cc930 0x7f0b7c0caf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.978 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b8321d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b7c0c9640 con 0x7f0b7c0cab40 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b8321d700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b7c0cd530 con 0x7f0b7c0cc930 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b8321d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0ccd90 con 0x7f0b7c1597d0 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b817ba700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0b7c0cc930 0x7f0b7c0caf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b7bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0b7c0cab40 0x7f0b7c0cc220 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36686/0 (socket says 172.21.15.123:36686) 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.975+0000 7f0b7bfff700 1 -- 172.21.15.123:0/2796556205 learned_addr learned my addr 172.21.15.123:0/2796556205 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b80fb9700 1 --2- 172.21.15.123:0/2796556205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c1597d0 0x7f0b7c068770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b817ba700 1 -- 172.21.15.123:0/2796556205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c1597d0 msgr2=0x7f0b7c068770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b817ba700 1 --2- 172.21.15.123:0/2796556205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c1597d0 0x7f0b7c068770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b817ba700 1 -- 172.21.15.123:0/2796556205 >> v1:172.21.15.123:6789/0 conn(0x7f0b7c0cab40 legacy=0x7f0b7c0cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.979 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b817ba700 1 -- 172.21.15.123:0/2796556205 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b7c0cb4d0 con 0x7f0b7c0cc930 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b7b7fe700 1 -- 172.21.15.123:0/2796556205 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1951844346 0 0) 0x7f0b7c0c9640 con 0x7f0b7c0cab40 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b817ba700 1 --2- 172.21.15.123:0/2796556205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c0caf90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009e60 tx=0x7f0b74009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d91a2ecc27879bcc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b7b7fe700 1 -- 172.21.15.123:0/2796556205 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b7400e070 con 0x7f0b7c0cc930 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b7b7fe700 1 -- 172.21.15.123:0/2796556205 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b7400bc30 con 0x7f0b7c0cc930 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.976+0000 7f0b7b7fe700 1 -- 172.21.15.123:0/2796556205 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b74012430 con 0x7f0b7c0cc930 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 -- 172.21.15.123:0/2796556205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 msgr2=0x7f0b7c0caf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2796556205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c0caf90 secure :-1 s=READY pgs=524 cs=0 l=1 rev1=1 crypto rx=0x7f0b74009e60 tx=0x7f0b74009780 comp rx=0 tx=0).stop 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 -- 172.21.15.123:0/2796556205 shutdown_connections 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2796556205 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c0caf90 unknown :-1 s=CLOSED pgs=524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2796556205 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c1597d0 0x7f0b7c068770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 -- 172.21.15.123:0/2796556205 >> 172.21.15.123:0/2796556205 conn(0x7f0b7c097450 msgr2=0x7f0b7c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:57.980 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 -- 172.21.15.123:0/2796556205 shutdown_connections 2023-10-06T04:32:57.981 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.977+0000 7f0b8321d700 1 -- 172.21.15.123:0/2796556205 wait complete. 2023-10-06T04:32:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.978+0000 7f0b8321d700 1 Processor -- start 2023-10-06T04:32:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.978+0000 7f0b8321d700 1 -- start start 2023-10-06T04:32:57.983 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b8321d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c0cab40 0x7f0b7c1695b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b8321d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c16db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b8321d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c1597d0 0x7f0b7c069350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:57.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b8321d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c169af0 con 0x7f0b7c0cc930 2023-10-06T04:32:57.984 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b8321d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0c1f40 con 0x7f0b7c1597d0 2023-10-06T04:32:58.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b8321d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b7c0c21d0 con 0x7f0b7c0cab40 2023-10-06T04:32:58.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c16db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c16db20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33468/0 (socket says 172.21.15.123:33468) 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 -- 172.21.15.123:0/2203977799 learned_addr learned my addr 172.21.15.123:0/2203977799 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b80fb9700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c0cab40 0x7f0b7c1695b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 -- 172.21.15.123:0/2203977799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c0cab40 msgr2=0x7f0b7c1695b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c0cab40 0x7f0b7c1695b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 -- 172.21.15.123:0/2203977799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c1597d0 msgr2=0x7f0b7c069350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c1597d0 0x7f0b7c069350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 -- 172.21.15.123:0/2203977799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b74009a20 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b7bfff700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c16db20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c5e0 tx=0x7f0b68001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b78ff9700 1 -- 172.21.15.123:0/2203977799 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b68005b30 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b78ff9700 1 -- 172.21.15.123:0/2203977799 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b68005c90 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b78ff9700 1 -- 172.21.15.123:0/2203977799 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b680072e0 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.979+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b7c0b29e0 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.980+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b7c16e0e0 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.980+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7c0b2b70 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.981+0000 7f0b78ff9700 1 -- 172.21.15.123:0/2203977799 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0b6801b3b0 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.981+0000 7f0b78ff9700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b60066f30 0x7f0b600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.981+0000 7f0b78ff9700 1 -- 172.21.15.123:0/2203977799 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f0b68083ce0 con 0x7f0b7c0cc930 2023-10-06T04:32:58.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.981+0000 7f0b80fb9700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b60066f30 0x7f0b600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.982+0000 7f0b80fb9700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b60066f30 0x7f0b600693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c153e40 tx=0x7f0b70008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:57.985+0000 7f0b78ff9700 1 -- 172.21.15.123:0/2203977799 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0b6804b5a0 con 0x7f0b7c0cc930 2023-10-06T04:32:58.165 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.164+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0b7c06d840 con 0x7f0b7c0cc930 2023-10-06T04:32:58.165 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.164+0000 7f0b78ff9700 1 -- 172.21.15.123:0/2203977799 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v42) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0b68017020 con 0x7f0b7c0cc930 2023-10-06T04:32:58.166 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:32:58.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b60066f30 msgr2=0x7f0b600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.167 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b60066f30 0x7f0b600693e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7f0b7c153e40 tx=0x7f0b70008040 comp rx=0 tx=0).stop 2023-10-06T04:32:58.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 msgr2=0x7f0b7c16db20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c16db20 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f0b6800c5e0 tx=0x7f0b68001070 comp rx=0 tx=0).stop 2023-10-06T04:32:58.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 shutdown_connections 2023-10-06T04:32:58.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0b60066f30 0x7f0b600693e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0b7c1597d0 0x7f0b7c069350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0b7c0cab40 0x7f0b7c1695b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.168 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 --2- 172.21.15.123:0/2203977799 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0b7c0cc930 0x7f0b7c16db20 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 >> 172.21.15.123:0/2203977799 conn(0x7f0b7c097450 msgr2=0x7f0b7c154500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:58.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 shutdown_connections 2023-10-06T04:32:58.169 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.166+0000 7f0b8321d700 1 -- 172.21.15.123:0/2203977799 wait complete. 2023-10-06T04:32:58.179 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:32:58.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.299+0000 7f233030b700 1 Processor -- start 2023-10-06T04:32:58.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.299+0000 7f233030b700 1 -- start start 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f233030b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2328157430 0x7f23280ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f233030b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23280caf00 0x7f23280c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f233030b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280c8f80 0x7f232815af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f233030b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2328157800 con 0x7f23280caf00 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f233030b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23280c9d70 con 0x7f2328157430 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f233030b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2328159200 con 0x7f23280c8f80 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23280caf00 0x7f23280c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23280caf00 0x7f23280c8a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33484/0 (socket says 172.21.15.123:33484) 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 -- 172.21.15.123:0/1400008448 learned_addr learned my addr 172.21.15.123:0/1400008448 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:58.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232e8a8700 1 --2- 172.21.15.123:0/1400008448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280c8f80 0x7f232815af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232e0a7700 1 --2- 172.21.15.123:0/1400008448 >> v2:172.21.15.177:3300/0 conn(0x7f2328157430 0x7f23280ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 -- 172.21.15.123:0/1400008448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280c8f80 msgr2=0x7f232815af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 --2- 172.21.15.123:0/1400008448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280c8f80 0x7f232815af10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 -- 172.21.15.123:0/1400008448 >> v2:172.21.15.177:3300/0 conn(0x7f2328157430 msgr2=0x7f23280ca9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 --2- 172.21.15.123:0/1400008448 >> v2:172.21.15.177:3300/0 conn(0x7f2328157430 0x7f23280ca9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 -- 172.21.15.123:0/1400008448 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23280cb3a0 con 0x7f23280caf00 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232e8a8700 1 --2- 172.21.15.123:0/1400008448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280c8f80 0x7f232815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232e0a7700 1 --2- 172.21.15.123:0/1400008448 >> v2:172.21.15.177:3300/0 conn(0x7f2328157430 0x7f23280ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d8a6700 1 --2- 172.21.15.123:0/1400008448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23280caf00 0x7f23280c8a40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f232806df50 tx=0x7f2318009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ed71b7b7d513de server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d0a5700 1 -- 172.21.15.123:0/1400008448 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800b860 con 0x7f23280caf00 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d0a5700 1 -- 172.21.15.123:0/1400008448 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f231800b9c0 con 0x7f23280caf00 2023-10-06T04:32:58.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.300+0000 7f232d0a5700 1 -- 172.21.15.123:0/1400008448 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f231800bd60 con 0x7f23280caf00 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 -- 172.21.15.123:0/1400008448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23280caf00 msgr2=0x7f23280c8a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 --2- 172.21.15.123:0/1400008448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23280caf00 0x7f23280c8a40 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f232806df50 tx=0x7f2318009250 comp rx=0 tx=0).stop 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 -- 172.21.15.123:0/1400008448 shutdown_connections 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 --2- 172.21.15.123:0/1400008448 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280c8f80 0x7f232815af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 --2- 172.21.15.123:0/1400008448 >> v2:172.21.15.177:3300/0 conn(0x7f2328157430 0x7f23280ca9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 --2- 172.21.15.123:0/1400008448 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23280caf00 0x7f23280c8a40 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 -- 172.21.15.123:0/1400008448 >> 172.21.15.123:0/1400008448 conn(0x7f2328097450 msgr2=0x7f2328067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 -- 172.21.15.123:0/1400008448 shutdown_connections 2023-10-06T04:32:58.305 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.301+0000 7f233030b700 1 -- 172.21.15.123:0/1400008448 wait complete. 2023-10-06T04:32:58.307 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.302+0000 7f233030b700 1 Processor -- start 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.302+0000 7f233030b700 1 -- start start 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f233030b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280c8f80 0x7f23280cd280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f233030b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280caf00 0x7f23280ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f233030b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2328157430 0x7f23280c2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f233030b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280ceab0 con 0x7f2328157430 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f233030b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23280c5410 con 0x7f23280c8f80 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f233030b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23280ce7e0 con 0x7f23280caf00 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2328157430 0x7f23280c2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2328157430 0x7f23280c2b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33498/0 (socket says 172.21.15.123:33498) 2023-10-06T04:32:58.308 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 -- 172.21.15.123:0/1280666602 learned_addr learned my addr 172.21.15.123:0/1280666602 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:58.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 -- 172.21.15.123:0/1280666602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280caf00 msgr2=0x7f23280ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280caf00 0x7f23280ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 -- 172.21.15.123:0/1280666602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280c8f80 msgr2=0x7f23280cd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280c8f80 0x7f23280cd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 -- 172.21.15.123:0/1280666602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23180094f0 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.303+0000 7f232e8a8700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2328157430 0x7f23280c2b70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2320009fb0 tx=0x7f2320009a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.304+0000 7f231e7fc700 1 -- 172.21.15.123:0/1280666602 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f232000dd90 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.304+0000 7f231e7fc700 1 -- 172.21.15.123:0/1280666602 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f232000f040 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.304+0000 7f231e7fc700 1 -- 172.21.15.123:0/1280666602 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2320015670 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.304+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23280c56b0 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.304+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23280c5c70 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.305+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23280c5e90 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.305+0000 7f231e7fc700 1 -- 172.21.15.123:0/1280666602 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f232001e3b0 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.306+0000 7f231e7fc700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f230c066f30 0x7f230c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.306+0000 7f231e7fc700 1 -- 172.21.15.123:0/1280666602 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2320084e20 con 0x7f2328157430 2023-10-06T04:32:58.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.306+0000 7f232e0a7700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f230c066f30 0x7f230c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.306+0000 7f232e0a7700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f230c066f30 0x7f230c0693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23240099c0 tx=0x7f2324008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.310+0000 7f231e7fc700 1 -- 172.21.15.123:0/1280666602 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2320051d10 con 0x7f2328157430 2023-10-06T04:32:58.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.489+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f232806d840 con 0x7f2328157430 2023-10-06T04:32:58.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.490+0000 7f231e7fc700 1 -- 172.21.15.123:0/1280666602 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v42) v1 ==== 92+0+10 (secure 0 0 0) 0x7f232001a0a0 con 0x7f2328157430 2023-10-06T04:32:58.491 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:32:58.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f230c066f30 msgr2=0x7f230c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f230c066f30 0x7f230c0693e0 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f23240099c0 tx=0x7f2324008040 comp rx=0 tx=0).stop 2023-10-06T04:32:58.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2328157430 msgr2=0x7f23280c2b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2328157430 0x7f23280c2b70 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f2320009fb0 tx=0x7f2320009a50 comp rx=0 tx=0).stop 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 shutdown_connections 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f230c066f30 0x7f230c0693e0 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2328157430 0x7f23280c2b70 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f23280c8f80 0x7f23280cd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 --2- 172.21.15.123:0/1280666602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f23280caf00 0x7f23280ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 >> 172.21.15.123:0/1280666602 conn(0x7f2328097450 msgr2=0x7f232806ccb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 shutdown_connections 2023-10-06T04:32:58.494 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.492+0000 7f233030b700 1 -- 172.21.15.123:0/1280666602 wait complete. 2023-10-06T04:32:58.505 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-10-06T04:32:58.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.624+0000 7f9b01a7f700 1 Processor -- start 2023-10-06T04:32:58.625 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.624+0000 7f9b01a7f700 1 -- start start 2023-10-06T04:32:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9b01a7f700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f9afc068670 0x7f9afc068ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9b01a7f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc068ff0 con 0x7f9afc15a1e0 2023-10-06T04:32:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9b01a7f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9afc06b9c0 con 0x7f9afc06a860 2023-10-06T04:32:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9b01a7f700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9afc069aa0 con 0x7f9afc068670 2023-10-06T04:32:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9afb7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f9afc15a1e0 0x7f9afc15a5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36698/0 (socket says 172.21.15.123:36698) 2023-10-06T04:32:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9afb7fe700 1 -- 172.21.15.123:0/710456808 learned_addr learned my addr 172.21.15.123:0/710456808 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:58.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9afbfff700 1 --2- 172.21.15.123:0/710456808 >> v2:172.21.15.177:3300/0 conn(0x7f9afc068670 0x7f9afc068ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1843331820 0 0) 0x7f9afc068ff0 con 0x7f9afc15a1e0 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.625+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9ae4003570 con 0x7f9afc15a1e0 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1467907607 0 0) 0x7f9ae4003570 con 0x7f9afc15a1e0 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 >> v2:172.21.15.177:3300/0 conn(0x7f9afc068670 msgr2=0x7f9afc068ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 --2- 172.21.15.123:0/710456808 >> v2:172.21.15.177:3300/0 conn(0x7f9afc068670 0x7f9afc068ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 >> v1:172.21.15.177:6789/0 conn(0x7f9afc06a860 legacy=0x7f9afc06af90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc069d80 con 0x7f9afc15a1e0 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f9aec001470 con 0x7f9afc15a1e0 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 >> v1:172.21.15.123:6789/0 conn(0x7f9afc15a1e0 legacy=0x7f9afc15a5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ae4003a60 0x7f9ae4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.630 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9ae4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ae40051c0 0x7f9ae400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc069aa0 con 0x7f9ae4004810 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae400c0d0 con 0x7f9ae40051c0 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ae400c290 con 0x7f9ae4003a60 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afaffd700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9ae4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afb7fe700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ae4003a60 0x7f9ae4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afbfff700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ae40051c0 0x7f9ae400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afaffd700 1 -- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ae4003a60 msgr2=0x7f9ae4003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afaffd700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9ae4003a60 0x7f9ae4003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afaffd700 1 -- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ae40051c0 msgr2=0x7f9ae400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afaffd700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9ae40051c0 0x7f9ae400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afbfff700 1 -- 172.21.15.123:0/710456808 reap_dead start 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afaffd700 1 -- 172.21.15.123:0/710456808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ae4003570 con 0x7f9ae4004810 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afaffd700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9ae4004c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9af00018b0 tx=0x7f9af0007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=938d2b85e41689a1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0012420 con 0x7f9ae4004810 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af00125a0 con 0x7f9ae4004810 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.626+0000 7f9afa7fc700 1 -- 172.21.15.123:0/710456808 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af0012960 con 0x7f9ae4004810 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.627+0000 7f9b01a7f700 1 -- 172.21.15.123:0/710456808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 msgr2=0x7f9ae4004c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.627+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9ae4004c80 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f9af00018b0 tx=0x7f9af0007d30 comp rx=0 tx=0).stop 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.627+0000 7f9b01a7f700 1 -- 172.21.15.123:0/710456808 shutdown_connections 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.627+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/710456808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9ae4004c80 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.627+0000 7f9b01a7f700 1 -- 172.21.15.123:0/710456808 >> 172.21.15.123:0/710456808 conn(0x7f9afc0c9ca0 msgr2=0x7f9afc06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.627+0000 7f9b01a7f700 1 -- 172.21.15.123:0/710456808 shutdown_connections 2023-10-06T04:32:58.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.627+0000 7f9b01a7f700 1 -- 172.21.15.123:0/710456808 wait complete. 2023-10-06T04:32:58.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.629+0000 7f9b01a7f700 1 Processor -- start 2023-10-06T04:32:58.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.629+0000 7f9b01a7f700 1 -- start start 2023-10-06T04:32:58.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9afc0c2960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc15a5b0 0x7f9afc145c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9afc0c3a90 0x7f9afc0c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc0c4440 con 0x7f9ae4004810 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc15aa20 con 0x7f9afc15a5b0 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9afc15ace0 con 0x7f9afc0c3a90 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9afc0c2960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9afc0c2960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33508/0 (socket says 172.21.15.123:33508) 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 -- 172.21.15.123:0/3299540455 learned_addr learned my addr 172.21.15.123:0/3299540455 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 -- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9afc0c3a90 msgr2=0x7f9afc0c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afbfff700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9afc0c3a90 0x7f9afc0c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afaffd700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc15a5b0 0x7f9afc145c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9afc0c3a90 0x7f9afc0c3f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 -- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc15a5b0 msgr2=0x7f9afc145c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc15a5b0 0x7f9afc145c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 -- 172.21.15.123:0/3299540455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc15afc0 con 0x7f9ae4004810 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9afb7fe700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9afc0c2960 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9aec007310 tx=0x7f9aec001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9ae37fe700 1 -- 172.21.15.123:0/3299540455 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec018030 con 0x7f9ae4004810 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9ae37fe700 1 -- 172.21.15.123:0/3299540455 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aec007780 con 0x7f9ae4004810 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9afc15b2a0 con 0x7f9ae4004810 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9ae37fe700 1 -- 172.21.15.123:0/3299540455 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aec00fb40 con 0x7f9ae4004810 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.630+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9afc15b860 con 0x7f9ae4004810 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.631+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9afc04bb80 con 0x7f9ae4004810 2023-10-06T04:32:58.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.632+0000 7f9ae37fe700 1 -- 172.21.15.123:0/3299540455 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f9aec01b070 con 0x7f9ae4004810 2023-10-06T04:32:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.632+0000 7f9ae37fe700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9ae8066f30 0x7f9ae80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.632+0000 7f9ae37fe700 1 -- 172.21.15.123:0/3299540455 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f9aec019070 con 0x7f9ae4004810 2023-10-06T04:32:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.632+0000 7f9afaffd700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9ae8066f30 0x7f9ae80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.632+0000 7f9afaffd700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9ae8066f30 0x7f9ae80693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9afc159a30 tx=0x7f9af0015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.636+0000 7f9ae37fe700 1 -- 172.21.15.123:0/3299540455 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f9aec051640 con 0x7f9ae4004810 2023-10-06T04:32:58.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.816+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f9afc0636b0 con 0x7f9ae4004810 2023-10-06T04:32:58.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.816+0000 7f9ae37fe700 1 -- 172.21.15.123:0/3299540455 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 104+0+11 (secure 0 0 0) 0x7f9aec054c60 con 0x7f9ae4004810 2023-10-06T04:32:58.818 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:58.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9ae8066f30 msgr2=0x7f9ae80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9ae8066f30 0x7f9ae80693e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7f9afc159a30 tx=0x7f9af0015000 comp rx=0 tx=0).stop 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 msgr2=0x7f9afc0c2960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9afc0c2960 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f9aec007310 tx=0x7f9aec001e50 comp rx=0 tx=0).stop 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 shutdown_connections 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f9ae8066f30 0x7f9ae80693e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f9ae4004810 0x7f9afc0c2960 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f9afc0c3a90 0x7f9afc0c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 --2- 172.21.15.123:0/3299540455 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f9afc15a5b0 0x7f9afc145c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 >> 172.21.15.123:0/3299540455 conn(0x7f9afc0c9ca0 msgr2=0x7f9afc0b1640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 shutdown_connections 2023-10-06T04:32:58.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.818+0000 7f9b01a7f700 1 -- 172.21.15.123:0/3299540455 wait complete. 2023-10-06T04:32:58.831 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:32:58.952 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.950+0000 7f4861874700 1 Processor -- start 2023-10-06T04:32:58.952 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.950+0000 7f4861874700 1 -- start start 2023-10-06T04:32:58.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f4861874700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f485c0c8340 0x7f485c0a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.954 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f4861874700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f485c068d80 0x7f485c0c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f4861874700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f485c069aa0 con 0x7f485c0c8710 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f4861874700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c06a340 con 0x7f485c068d80 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f4861874700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f485c0aa6f0 con 0x7f485c0c8340 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f485a7fc700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f485c0c8710 0x7f485c068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36708/0 (socket says 172.21.15.123:36708) 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f485a7fc700 1 -- 172.21.15.123:0/2544629133 learned_addr learned my addr 172.21.15.123:0/2544629133 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.951+0000 7f485b7fe700 1 --2- 172.21.15.123:0/2544629133 >> v2:172.21.15.123:3300/0 conn(0x7f485c068d80 0x7f485c0c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f4859ffb700 1 -- 172.21.15.123:0/2544629133 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1385502918 0 0) 0x7f485c069aa0 con 0x7f485c0c8710 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f4859ffb700 1 -- 172.21.15.123:0/2544629133 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4844003570 con 0x7f485c0c8710 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f485affd700 1 --2- 172.21.15.123:0/2544629133 >> v2:172.21.15.177:3300/0 conn(0x7f485c0c8340 0x7f485c0a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f485b7fe700 1 -- 172.21.15.123:0/2544629133 >> v2:172.21.15.177:3300/0 conn(0x7f485c0c8340 msgr2=0x7f485c0a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f485b7fe700 1 --2- 172.21.15.123:0/2544629133 >> v2:172.21.15.177:3300/0 conn(0x7f485c0c8340 0x7f485c0a9df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.955 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f485b7fe700 1 -- 172.21.15.123:0/2544629133 >> v1:172.21.15.123:6789/0 conn(0x7f485c0c8710 legacy=0x7f485c068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f485b7fe700 1 -- 172.21.15.123:0/2544629133 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f485c069cf0 con 0x7f485c068d80 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f485affd700 1 --2- 172.21.15.123:0/2544629133 >> v2:172.21.15.177:3300/0 conn(0x7f485c0c8340 0x7f485c0a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f485b7fe700 1 --2- 172.21.15.123:0/2544629133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c068d80 0x7f485c0c8b60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f485c06d1c0 tx=0x7f4854009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b7afddbc17607f8 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f4859ffb700 1 -- 172.21.15.123:0/2544629133 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f485400e070 con 0x7f485c068d80 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f4859ffb700 1 -- 172.21.15.123:0/2544629133 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f485400bb60 con 0x7f485c068d80 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.952+0000 7f4859ffb700 1 -- 172.21.15.123:0/2544629133 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4854012430 con 0x7f485c068d80 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 -- 172.21.15.123:0/2544629133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c068d80 msgr2=0x7f485c0c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 --2- 172.21.15.123:0/2544629133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c068d80 0x7f485c0c8b60 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f485c06d1c0 tx=0x7f4854009780 comp rx=0 tx=0).stop 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 -- 172.21.15.123:0/2544629133 shutdown_connections 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 --2- 172.21.15.123:0/2544629133 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c068d80 0x7f485c0c8b60 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 --2- 172.21.15.123:0/2544629133 >> v2:172.21.15.177:3300/0 conn(0x7f485c0c8340 0x7f485c0a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.956 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 -- 172.21.15.123:0/2544629133 >> 172.21.15.123:0/2544629133 conn(0x7f485c0c9ca0 msgr2=0x7f485c06da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:58.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 -- 172.21.15.123:0/2544629133 shutdown_connections 2023-10-06T04:32:58.957 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.953+0000 7f4861874700 1 -- 172.21.15.123:0/2544629133 wait complete. 2023-10-06T04:32:58.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.954+0000 7f4861874700 1 Processor -- start 2023-10-06T04:32:58.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.954+0000 7f4861874700 1 -- start start 2023-10-06T04:32:58.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f485c068d80 0x7f485c157730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c0c8340 0x7f485c0c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485c0c8710 0x7f485c0c5bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c159e30 con 0x7f485c0c8340 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f485c157c70 con 0x7f485c0c8710 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f485c0ae3a0 con 0x7f485c068d80 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c0c8340 0x7f485c0c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c0c8340 0x7f485c0c37f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33532/0 (socket says 172.21.15.123:33532) 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 -- 172.21.15.123:0/1073182221 learned_addr learned my addr 172.21.15.123:0/1073182221 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485b7fe700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485c0c8710 0x7f485c0c5bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485affd700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f485c068d80 0x7f485c157730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.960 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 -- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f485c068d80 msgr2=0x7f485c157730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f485c068d80 0x7f485c157730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 -- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485c0c8710 msgr2=0x7f485c0c5bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485c0c8710 0x7f485c0c5bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 -- 172.21.15.123:0/1073182221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4854009a20 con 0x7f485c0c8340 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485affd700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f485c068d80 0x7f485c157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f485a7fc700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c0c8340 0x7f485c0c37f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c590 tx=0x7f484c007e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4842ffd700 1 -- 172.21.15.123:0/1073182221 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c005b30 con 0x7f485c0c8340 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4842ffd700 1 -- 172.21.15.123:0/1073182221 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f484c005cb0 con 0x7f485c0c8340 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f485c0ae680 con 0x7f485c0c8340 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4842ffd700 1 -- 172.21.15.123:0/1073182221 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f484c007320 con 0x7f485c0c8340 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.955+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f485c0aec40 con 0x7f485c0c8340 2023-10-06T04:32:58.961 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.956+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f485c159f70 con 0x7f485c0c8340 2023-10-06T04:32:58.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.957+0000 7f4842ffd700 1 -- 172.21.15.123:0/1073182221 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f484c01b3b0 con 0x7f485c0c8340 2023-10-06T04:32:58.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.957+0000 7f4842ffd700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4848066f30 0x7f48480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:58.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.957+0000 7f4842ffd700 1 -- 172.21.15.123:0/1073182221 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f484c083e70 con 0x7f485c0c8340 2023-10-06T04:32:58.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.957+0000 7f485affd700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4848066f30 0x7f48480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:58.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.958+0000 7f485affd700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4848066f30 0x7f48480693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4850008b20 tx=0x7f4850008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:58.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:58.961+0000 7f4842ffd700 1 -- 172.21.15.123:0/1073182221 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f484c050d60 con 0x7f485c0c8340 2023-10-06T04:32:59.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.141+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f485c069fd0 con 0x7f485c0c8340 2023-10-06T04:32:59.143 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.142+0000 7f4842ffd700 1 -- 172.21.15.123:0/1073182221 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v42) v1 ==== 100+0+11 (secure 0 0 0) 0x7f484c017020 con 0x7f485c0c8340 2023-10-06T04:32:59.144 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:32:59.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4848066f30 msgr2=0x7f48480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4848066f30 0x7f48480693e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f4850008b20 tx=0x7f4850008040 comp rx=0 tx=0).stop 2023-10-06T04:32:59.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c0c8340 msgr2=0x7f485c0c37f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c0c8340 0x7f485c0c37f0 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f484c00c590 tx=0x7f484c007e00 comp rx=0 tx=0).stop 2023-10-06T04:32:59.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 shutdown_connections 2023-10-06T04:32:59.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4848066f30 0x7f48480693e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f485c0c8710 0x7f485c0c5bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f485c068d80 0x7f485c157730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 --2- 172.21.15.123:0/1073182221 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f485c0c8340 0x7f485c0c37f0 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 >> 172.21.15.123:0/1073182221 conn(0x7f485c0c9ca0 msgr2=0x7f485c158ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:59.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 shutdown_connections 2023-10-06T04:32:59.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.144+0000 7f4861874700 1 -- 172.21.15.123:0/1073182221 wait complete. 2023-10-06T04:32:59.157 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get recovery_fs_metadata pg_num 2023-10-06T04:32:59.277 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.276+0000 7fd47608b700 1 Processor -- start 2023-10-06T04:32:59.278 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.276+0000 7fd47608b700 1 -- start start 2023-10-06T04:32:59.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd47608b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd4700c8340 0x7fd4700a9df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd47608b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8710 0x7fd4700c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd47608b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd470068670 0x7fd470068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd47608b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd470069050 con 0x7fd470068670 2023-10-06T04:32:59.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd47608b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd470069aa0 con 0x7fd4700c8340 2023-10-06T04:32:59.280 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd47608b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd47006a340 con 0x7fd4700c8710 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd470068670 0x7fd470068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd470068670 0x7fd470068b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33534/0 (socket says 172.21.15.123:33534) 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 -- 172.21.15.123:0/1156160929 learned_addr learned my addr 172.21.15.123:0/1156160929 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46effd700 1 --2- 172.21.15.123:0/1156160929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8710 0x7fd4700c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 -- 172.21.15.123:0/1156160929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8710 msgr2=0x7fd4700c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 --2- 172.21.15.123:0/1156160929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8710 0x7fd4700c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 -- 172.21.15.123:0/1156160929 >> v2:172.21.15.177:3300/0 conn(0x7fd4700c8340 msgr2=0x7fd4700a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46f7fe700 1 --2- 172.21.15.123:0/1156160929 >> v2:172.21.15.177:3300/0 conn(0x7fd4700c8340 0x7fd4700a9df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 --2- 172.21.15.123:0/1156160929 >> v2:172.21.15.177:3300/0 conn(0x7fd4700c8340 0x7fd4700a9df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 -- 172.21.15.123:0/1156160929 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4700aa6f0 con 0x7fd470068670 2023-10-06T04:32:59.281 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.277+0000 7fd46ffff700 1 --2- 172.21.15.123:0/1156160929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd470068670 0x7fd470068b10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd468009f20 tx=0x7fd468009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4519ce4aae070e83 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd46e7fc700 1 -- 172.21.15.123:0/1156160929 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd46800e070 con 0x7fd470068670 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd46e7fc700 1 -- 172.21.15.123:0/1156160929 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd46800ba40 con 0x7fd470068670 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd46e7fc700 1 -- 172.21.15.123:0/1156160929 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680092e0 con 0x7fd470068670 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 -- 172.21.15.123:0/1156160929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd470068670 msgr2=0x7fd470068b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 --2- 172.21.15.123:0/1156160929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd470068670 0x7fd470068b10 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fd468009f20 tx=0x7fd468009840 comp rx=0 tx=0).stop 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 -- 172.21.15.123:0/1156160929 shutdown_connections 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 --2- 172.21.15.123:0/1156160929 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd470068670 0x7fd470068b10 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 --2- 172.21.15.123:0/1156160929 >> v2:172.21.15.177:3300/0 conn(0x7fd4700c8340 0x7fd4700a9df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 --2- 172.21.15.123:0/1156160929 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8710 0x7fd4700c8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 -- 172.21.15.123:0/1156160929 >> 172.21.15.123:0/1156160929 conn(0x7fd4700c9ca0 msgr2=0x7fd47006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 -- 172.21.15.123:0/1156160929 shutdown_connections 2023-10-06T04:32:59.282 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.278+0000 7fd47608b700 1 -- 172.21.15.123:0/1156160929 wait complete. 2023-10-06T04:32:59.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.279+0000 7fd47608b700 1 Processor -- start 2023-10-06T04:32:59.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.279+0000 7fd47608b700 1 -- start start 2023-10-06T04:32:59.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd47608b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470068670 0x7fd4700ae1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd47608b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8340 0x7fd4700c0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd47608b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4700c8710 0x7fd4700c0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd47608b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4701584c0 con 0x7fd4700c8710 2023-10-06T04:32:59.285 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd47608b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700ae730 con 0x7fd470068670 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd47608b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4700ae960 con 0x7fd4700c8340 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd46ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4700c8710 0x7fd4700c0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd46ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4700c8710 0x7fd4700c0640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33546/0 (socket says 172.21.15.123:33546) 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd46ffff700 1 -- 172.21.15.123:0/1573508520 learned_addr learned my addr 172.21.15.123:0/1573508520 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd46f7fe700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470068670 0x7fd4700ae1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.280+0000 7fd46effd700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8340 0x7fd4700c0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd46ffff700 1 -- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8340 msgr2=0x7fd4700c0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd46ffff700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8340 0x7fd4700c0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd46ffff700 1 -- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470068670 msgr2=0x7fd4700ae1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.286 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd46ffff700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470068670 0x7fd4700ae1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd46ffff700 1 -- 172.21.15.123:0/1573508520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd468009ae0 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd46ffff700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4700c8710 0x7fd4700c0640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd47006d190 tx=0x7fd468018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd4577fe700 1 -- 172.21.15.123:0/1573508520 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd468008ce0 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd4577fe700 1 -- 172.21.15.123:0/1573508520 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd468008e60 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4700c2350 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd4577fe700 1 -- 172.21.15.123:0/1573508520 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4680266b0 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.281+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4700c1d00 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.282+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4700c24e0 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.282+0000 7fd4577fe700 1 -- 172.21.15.123:0/1573508520 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd468003040 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.283+0000 7fd4577fe700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd45c066f30 0x7fd45c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.283+0000 7fd4577fe700 1 -- 172.21.15.123:0/1573508520 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fd4680868f0 con 0x7fd4700c8710 2023-10-06T04:32:59.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.283+0000 7fd46f7fe700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd45c066f30 0x7fd45c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.283+0000 7fd46f7fe700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd45c066f30 0x7fd45c0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd47015ccd0 tx=0x7fd464008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.289 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.288+0000 7fd4577fe700 1 -- 172.21.15.123:0/1573508520 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd4680537e0 con 0x7fd4700c8710 2023-10-06T04:32:59.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.469+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd4700ca0a0 con 0x7fd4700c8710 2023-10-06T04:32:59.470 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.469+0000 7fd4577fe700 1 -- 172.21.15.123:0/1573508520 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"}]=0 v42) v1 ==== 109+0+11 (secure 0 0 0) 0x7fd468021020 con 0x7fd4700c8710 2023-10-06T04:32:59.470 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:32:59.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd45c066f30 msgr2=0x7fd45c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd45c066f30 0x7fd45c0693e0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fd47015ccd0 tx=0x7fd464008040 comp rx=0 tx=0).stop 2023-10-06T04:32:59.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4700c8710 msgr2=0x7fd4700c0640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4700c8710 0x7fd4700c0640 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fd47006d190 tx=0x7fd468018000 comp rx=0 tx=0).stop 2023-10-06T04:32:59.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 shutdown_connections 2023-10-06T04:32:59.473 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd45c066f30 0x7fd45c0693e0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd4700c8710 0x7fd4700c0640 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd470068670 0x7fd4700ae1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 --2- 172.21.15.123:0/1573508520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd4700c8340 0x7fd4700c0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 >> 172.21.15.123:0/1573508520 conn(0x7fd4700c9ca0 msgr2=0x7fd4700a98b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:59.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 shutdown_connections 2023-10-06T04:32:59.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.471+0000 7fd47608b700 1 -- 172.21.15.123:0/1573508520 wait complete. 2023-10-06T04:32:59.484 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:32:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.599+0000 7f181ca2f700 1 Processor -- start 2023-10-06T04:32:59.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.600+0000 7f181ca2f700 1 -- start start 2023-10-06T04:32:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.600+0000 7f181ca2f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18180688c0 0x7f18180c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181ca2f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1818069a30 con 0x7f1818068c90 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181ca2f700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f181806a890 con 0x7f1818158f70 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181ca2f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1818069c80 con 0x7f18180688c0 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181659c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f18180688c0 0x7f18180c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f1815d9b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1818068c90 0x7f181806e430 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36720/0 (socket says 172.21.15.123:36720) 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f1815d9b700 1 -- 172.21.15.123:0/2902049028 learned_addr learned my addr 172.21.15.123:0/2902049028 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181659c700 1 -- 172.21.15.123:0/2902049028 >> v1:172.21.15.177:6789/0 conn(0x7f1818158f70 legacy=0x7f18181593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181659c700 1 -- 172.21.15.123:0/2902049028 >> v1:172.21.15.123:6789/0 conn(0x7f1818068c90 legacy=0x7f181806e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181659c700 1 -- 172.21.15.123:0/2902049028 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1818069f60 con 0x7f18180688c0 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181559a700 1 -- 172.21.15.123:0/2902049028 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 778852395 0 0) 0x7f1818069a30 con 0x7f1818068c90 2023-10-06T04:32:59.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181659c700 1 --2- 172.21.15.123:0/2902049028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f18180c6dd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f180c0098e0 tx=0x7f180c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea2459dbc8810e41 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181559a700 1 -- 172.21.15.123:0/2902049028 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c009e80 con 0x7f18180688c0 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181559a700 1 -- 172.21.15.123:0/2902049028 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c00b860 con 0x7f18180688c0 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.601+0000 7f181559a700 1 -- 172.21.15.123:0/2902049028 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c00bbe0 con 0x7f18180688c0 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.602+0000 7f181ca2f700 1 -- 172.21.15.123:0/2902049028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 msgr2=0x7f18180c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.602+0000 7f181ca2f700 1 --2- 172.21.15.123:0/2902049028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f18180c6dd0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7f180c0098e0 tx=0x7f180c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.602+0000 7f181ca2f700 1 -- 172.21.15.123:0/2902049028 shutdown_connections 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.602+0000 7f181ca2f700 1 --2- 172.21.15.123:0/2902049028 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f18180c6dd0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.602+0000 7f181ca2f700 1 -- 172.21.15.123:0/2902049028 >> 172.21.15.123:0/2902049028 conn(0x7f18180c9c90 msgr2=0x7f181806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.602+0000 7f181ca2f700 1 -- 172.21.15.123:0/2902049028 shutdown_connections 2023-10-06T04:32:59.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.602+0000 7f181ca2f700 1 -- 172.21.15.123:0/2902049028 wait complete. 2023-10-06T04:32:59.608 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.603+0000 7f181ca2f700 1 Processor -- start 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.603+0000 7f181ca2f700 1 -- start start 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181ca2f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f1818145660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181ca2f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1818068c90 0x7f1818159db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181ca2f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1818158f70 0x7f18180c63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181ca2f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181815a430 con 0x7f18180688c0 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181ca2f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f181815a710 con 0x7f1818068c90 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181ca2f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f181815a9f0 con 0x7f1818158f70 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f1818145660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f1818145660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33566/0 (socket says 172.21.15.123:33566) 2023-10-06T04:32:59.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 -- 172.21.15.123:0/3352022347 learned_addr learned my addr 172.21.15.123:0/3352022347 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 -- 172.21.15.123:0/3352022347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1818158f70 msgr2=0x7f18180c63a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1818158f70 0x7f18180c63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 -- 172.21.15.123:0/3352022347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1818068c90 msgr2=0x7f1818159db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1818068c90 0x7f1818159db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 -- 172.21.15.123:0/3352022347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f180c009420 con 0x7f18180688c0 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181659c700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f1818145660 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f181806d230 tx=0x7f180c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f18067fc700 1 -- 172.21.15.123:0/3352022347 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c01f070 con 0x7f18180688c0 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f18067fc700 1 -- 172.21.15.123:0/3352022347 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f180c00b9d0 con 0x7f18180688c0 2023-10-06T04:32:59.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f18067fc700 1 -- 172.21.15.123:0/3352022347 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f180c025670 con 0x7f18180688c0 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.604+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f181815acd0 con 0x7f18180688c0 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.605+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1818144e80 con 0x7f18180688c0 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.605+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f181815b1f0 con 0x7f18180688c0 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.606+0000 7f18067fc700 1 -- 172.21.15.123:0/3352022347 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f180c023070 con 0x7f18180688c0 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.606+0000 7f18067fc700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f17f8066f30 0x7f17f80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.606+0000 7f1815d9b700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f17f8066f30 0x7f17f80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.606+0000 7f18067fc700 1 -- 172.21.15.123:0/3352022347 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f180c085f50 con 0x7f18180688c0 2023-10-06T04:32:59.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.607+0000 7f1815d9b700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f17f8066f30 0x7f17f80693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f18080011e0 tx=0x7f18080079d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.611+0000 7f18067fc700 1 -- 172.21.15.123:0/3352022347 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f181815b1f0 con 0x7f18180688c0 2023-10-06T04:32:59.833 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.832+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f181815b1f0 con 0x7f18180688c0 2023-10-06T04:32:59.834 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.833+0000 7f18067fc700 1 -- 172.21.15.123:0/3352022347 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5671 (secure 0 0 0) 0x7f181815b1f0 con 0x7f18180688c0 2023-10-06T04:32:59.834 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:32:59.834 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:32:59.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f17f8066f30 msgr2=0x7f17f80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f17f8066f30 0x7f17f80693e0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f18080011e0 tx=0x7f18080079d0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 msgr2=0x7f1818145660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f1818145660 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f181806d230 tx=0x7f180c017000 comp rx=0 tx=0).stop 2023-10-06T04:32:59.836 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 shutdown_connections 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f17f8066f30 0x7f17f80693e0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f18180688c0 0x7f1818145660 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1818158f70 0x7f18180c63a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 --2- 172.21.15.123:0/3352022347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1818068c90 0x7f1818159db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.834+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 >> 172.21.15.123:0/3352022347 conn(0x7f18180c9c90 msgr2=0x7f18181575e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.835+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 shutdown_connections 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.835+0000 7f181ca2f700 1 -- 172.21.15.123:0/3352022347 wait complete. 2023-10-06T04:32:59.837 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 26 2023-10-06T04:32:59.848 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:32:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.963+0000 7f24ebbb2700 1 Processor -- start 2023-10-06T04:32:59.965 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24ebbb2700 1 -- start start 2023-10-06T04:32:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24ebbb2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24e4067660 0x7f24e4067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24ebbb2700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e406f070 con 0x7f24e40c9570 2023-10-06T04:32:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24ebbb2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24e4067f70 con 0x7f24e40aa950 2023-10-06T04:32:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24ebbb2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24e406a750 con 0x7f24e4067660 2023-10-06T04:32:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24e994e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f24e4067660 0x7f24e4067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24e914d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f24e40c9570 0x7f24e406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:36730/0 (socket says 172.21.15.123:36730) 2023-10-06T04:32:59.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.964+0000 7f24e914d700 1 -- 172.21.15.123:0/1902633851 learned_addr learned my addr 172.21.15.123:0/1902633851 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e994e700 1 -- 172.21.15.123:0/1902633851 >> v1:172.21.15.177:6789/0 conn(0x7f24e40aa950 legacy=0x7f24e406e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e994e700 1 -- 172.21.15.123:0/1902633851 >> v1:172.21.15.123:6789/0 conn(0x7f24e40c9570 legacy=0x7f24e406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e994e700 1 -- 172.21.15.123:0/1902633851 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e406aa30 con 0x7f24e4067660 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e894c700 1 -- 172.21.15.123:0/1902633851 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 199142084 0 0) 0x7f24e406f070 con 0x7f24e40c9570 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e994e700 1 --2- 172.21.15.123:0/1902633851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4067a30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f24e00098e0 tx=0x7f24e00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=382e135ad139e63b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e894c700 1 -- 172.21.15.123:0/1902633851 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e0009e80 con 0x7f24e4067660 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e894c700 1 -- 172.21.15.123:0/1902633851 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e000b860 con 0x7f24e4067660 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.965+0000 7f24e894c700 1 -- 172.21.15.123:0/1902633851 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000bbe0 con 0x7f24e4067660 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.966+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1902633851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 msgr2=0x7f24e4067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.966+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1902633851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4067a30 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f24e00098e0 tx=0x7f24e00091a0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.966+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1902633851 shutdown_connections 2023-10-06T04:32:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.966+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1902633851 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4067a30 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.966+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1902633851 >> 172.21.15.123:0/1902633851 conn(0x7f24e40cda80 msgr2=0x7f24e40c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:32:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.966+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1902633851 shutdown_connections 2023-10-06T04:32:59.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.966+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1902633851 wait complete. 2023-10-06T04:32:59.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.967+0000 7f24ebbb2700 1 Processor -- start 2023-10-06T04:32:59.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.967+0000 7f24ebbb2700 1 -- start start 2023-10-06T04:32:59.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.967+0000 7f24ebbb2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4156f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.967+0000 7f24ebbb2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40aa950 0x7f24e40c0c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.967+0000 7f24ebbb2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e40c9570 0x7f24e40c11c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.967+0000 7f24ebbb2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40ac8c0 con 0x7f24e4067660 2023-10-06T04:32:59.972 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.967+0000 7f24ebbb2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40c2e70 con 0x7f24e40c9570 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24ebbb2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24e40c3150 con 0x7f24e40aa950 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4156f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4156f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:33580/0 (socket says 172.21.15.123:33580) 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 -- 172.21.15.123:0/1058325470 learned_addr learned my addr 172.21.15.123:0/1058325470 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 -- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40aa950 msgr2=0x7f24e40c0c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e914d700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40aa950 0x7f24e40c0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40aa950 0x7f24e40c0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 -- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e40c9570 msgr2=0x7f24e40c11c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24ea14f700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e40c9570 0x7f24e40c11c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e40c9570 0x7f24e40c11c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 -- 172.21.15.123:0/1058325470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24e0009420 con 0x7f24e4067660 2023-10-06T04:32:59.973 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24e994e700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4156f50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f24e406fd80 tx=0x7f24e0017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24d9ffb700 1 -- 172.21.15.123:0/1058325470 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e001f070 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24d9ffb700 1 -- 172.21.15.123:0/1058325470 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24e0011420 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24e40c3430 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24d9ffb700 1 -- 172.21.15.123:0/1058325470 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24e000bbe0 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.968+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24e4157490 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.969+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24e40aca30 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.970+0000 7f24d9ffb700 1 -- 172.21.15.123:0/1058325470 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f24e0011a50 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.970+0000 7f24d9ffb700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24c8066f30 0x7f24c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.970+0000 7f24d9ffb700 1 -- 172.21.15.123:0/1058325470 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f24e0085e10 con 0x7f24e4067660 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.970+0000 7f24e914d700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24c8066f30 0x7f24c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:32:59.974 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.970+0000 7f24e914d700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24c8066f30 0x7f24c80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24e415eb50 tx=0x7f24d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:32:59.975 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:32:59.974+0000 7f24d9ffb700 1 -- 172.21.15.123:0/1058325470 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f24e40aca30 con 0x7f24e4067660 2023-10-06T04:33:00.160 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.159+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f24e40aca30 con 0x7f24e4067660 2023-10-06T04:33:00.161 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.159+0000 7f24d9ffb700 1 -- 172.21.15.123:0/1058325470 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18770 (secure 0 0 0) 0x7f24e0015090 con 0x7f24e4067660 2023-10-06T04:33:00.161 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:00.161 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:38.586940+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:33:00.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24c8066f30 msgr2=0x7f24c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.163 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24c8066f30 0x7f24c80693e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f24e415eb50 tx=0x7f24d4009040 comp rx=0 tx=0).stop 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 msgr2=0x7f24e4156f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4156f50 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f24e406fd80 tx=0x7f24e0017000 comp rx=0 tx=0).stop 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 shutdown_connections 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f24c8066f30 0x7f24c80693e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f24e4067660 0x7f24e4156f50 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f24e40c9570 0x7f24e40c11c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 --2- 172.21.15.123:0/1058325470 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f24e40aa950 0x7f24e40c0c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 >> 172.21.15.123:0/1058325470 conn(0x7f24e40cda80 msgr2=0x7f24e40cab20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 shutdown_connections 2023-10-06T04:33:00.164 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.162+0000 7f24ebbb2700 1 -- 172.21.15.123:0/1058325470 wait complete. 2023-10-06T04:33:00.175 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:33:00.175 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:33:00.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.291+0000 7f2c38181700 1 Processor -- start 2023-10-06T04:33:00.293 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.291+0000 7f2c38181700 1 -- start start 2023-10-06T04:33:00.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.292+0000 7f2c38181700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f2c30068d80 0x7f2c300c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.295 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.292+0000 7f2c38181700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300c7c50 con 0x7f2c300c7460 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.292+0000 7f2c38181700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2c300c7f30 con 0x7f2c30155780 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.292+0000 7f2c38181700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2c300c8210 con 0x7f2c30068d80 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.292+0000 7f2c3671e700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2c300c7460 0x7f2c3006aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60498/0 (socket says 172.21.15.123:60498) 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c3671e700 1 -- 172.21.15.123:0/3870337198 learned_addr learned my addr 172.21.15.123:0/3870337198 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c3571c700 1 --2- 172.21.15.123:0/3870337198 >> v2:172.21.15.177:3300/0 conn(0x7f2c30068d80 0x7f2c300c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c34f1b700 1 -- 172.21.15.123:0/3870337198 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 796195866 0 0) 0x7f2c300c7c50 con 0x7f2c300c7460 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c34f1b700 1 -- 172.21.15.123:0/3870337198 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2c1c004680 con 0x7f2c300c7460 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c34f1b700 1 -- 172.21.15.123:0/3870337198 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2631860048 0 0) 0x7f2c1c004680 con 0x7f2c300c7460 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c3571c700 1 -- 172.21.15.123:0/3870337198 >> v1:172.21.15.177:6789/0 conn(0x7f2c30155780 legacy=0x7f2c30068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c3571c700 1 -- 172.21.15.123:0/3870337198 >> v1:172.21.15.123:6789/0 conn(0x7f2c300c7460 legacy=0x7f2c3006aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c3571c700 1 -- 172.21.15.123:0/3870337198 --> v2:172.21.15.177:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c300c84f0 con 0x7f2c30068d80 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c3571c700 1 --2- 172.21.15.123:0/3870337198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 0x7f2c300c6f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2c200098e0 tx=0x7f2c200091a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3d6d207f192d65aa server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c34f1b700 1 -- 172.21.15.123:0/3870337198 <== mon.1 v2:172.21.15.177:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c280017f0 con 0x7f2c30068d80 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c34f1b700 1 -- 172.21.15.123:0/3870337198 <== mon.1 v2:172.21.15.177:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c20011420 con 0x7f2c30068d80 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.293+0000 7f2c34f1b700 1 -- 172.21.15.123:0/3870337198 <== mon.1 v2:172.21.15.177:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c200117c0 con 0x7f2c30068d80 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.294+0000 7f2c38181700 1 -- 172.21.15.123:0/3870337198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 msgr2=0x7f2c300c6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.294+0000 7f2c38181700 1 --2- 172.21.15.123:0/3870337198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 0x7f2c300c6f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f2c200098e0 tx=0x7f2c200091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.294+0000 7f2c38181700 1 -- 172.21.15.123:0/3870337198 shutdown_connections 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.294+0000 7f2c38181700 1 --2- 172.21.15.123:0/3870337198 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 0x7f2c300c6f20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.294+0000 7f2c38181700 1 -- 172.21.15.123:0/3870337198 >> 172.21.15.123:0/3870337198 conn(0x7f2c300c9c90 msgr2=0x7f2c3006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.294+0000 7f2c38181700 1 -- 172.21.15.123:0/3870337198 shutdown_connections 2023-10-06T04:33:00.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.294+0000 7f2c38181700 1 -- 172.21.15.123:0/3870337198 wait complete. 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.295+0000 7f2c38181700 1 Processor -- start 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.295+0000 7f2c38181700 1 -- start start 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c38181700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 0x7f2c300af6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c38181700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300c7460 0x7f2c30150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c38181700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c30155780 0x7f2c300c5b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c38181700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300b0010 con 0x7f2c30155780 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c38181700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300b02f0 con 0x7f2c30068d80 2023-10-06T04:33:00.300 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c38181700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2c300b05d0 con 0x7f2c300c7460 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c30155780 0x7f2c300c5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c30155780 0x7f2c300c5b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53188/0 (socket says 172.21.15.123:53188) 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 -- 172.21.15.123:0/3399614409 learned_addr learned my addr 172.21.15.123:0/3399614409 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3571c700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300c7460 0x7f2c30150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c35f1d700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 0x7f2c300af6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 -- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300c7460 msgr2=0x7f2c30150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300c7460 0x7f2c30150c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 -- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 msgr2=0x7f2c300af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 0x7f2c300af6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 -- 172.21.15.123:0/3399614409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c20009420 con 0x7f2c30155780 2023-10-06T04:33:00.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c3671e700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c30155780 0x7f2c300c5b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800ccc0 tx=0x7f2c280018e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:00.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.296+0000 7f2c25ffb700 1 -- 172.21.15.123:0/3399614409 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800ce70 con 0x7f2c30155780 2023-10-06T04:33:00.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.297+0000 7f2c25ffb700 1 -- 172.21.15.123:0/3399614409 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c28004dc0 con 0x7f2c30155780 2023-10-06T04:33:00.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.297+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2c300b08b0 con 0x7f2c30155780 2023-10-06T04:33:00.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.297+0000 7f2c25ffb700 1 -- 172.21.15.123:0/3399614409 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c2800abb0 con 0x7f2c30155780 2023-10-06T04:33:00.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.297+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2c300afda0 con 0x7f2c30155780 2023-10-06T04:33:00.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.298+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2c3006cc70 con 0x7f2c30155780 2023-10-06T04:33:00.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.298+0000 7f2c25ffb700 1 -- 172.21.15.123:0/3399614409 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2c28007150 con 0x7f2c30155780 2023-10-06T04:33:00.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.298+0000 7f2c25ffb700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2c14066f30 0x7f2c140693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.299+0000 7f2c25ffb700 1 -- 172.21.15.123:0/3399614409 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f2c28084520 con 0x7f2c30155780 2023-10-06T04:33:00.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.299+0000 7f2c35f1d700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2c14066f30 0x7f2c140693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.299+0000 7f2c35f1d700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2c14066f30 0x7f2c140693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0017c0 tx=0x7f2c2c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:00.304 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.303+0000 7f2c25ffb700 1 -- 172.21.15.123:0/3399614409 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2c28051490 con 0x7f2c30155780 2023-10-06T04:33:00.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.523+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2c300c8860 con 0x7f2c30155780 2023-10-06T04:33:00.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.523+0000 7f2c25ffb700 1 -- 172.21.15.123:0/3399614409 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5671 (secure 0 0 0) 0x7f2c28054ab0 con 0x7f2c30155780 2023-10-06T04:33:00.525 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:00.525 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:33:00.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2c14066f30 msgr2=0x7f2c140693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2c14066f30 0x7f2c140693e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2c2c0017c0 tx=0x7f2c2c009040 comp rx=0 tx=0).stop 2023-10-06T04:33:00.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c30155780 msgr2=0x7f2c300c5b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c30155780 0x7f2c300c5b80 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f2c2800ccc0 tx=0x7f2c280018e0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 shutdown_connections 2023-10-06T04:33:00.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f2c14066f30 0x7f2c140693e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2c30155780 0x7f2c300c5b80 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2c30068d80 0x7f2c300af6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 --2- 172.21.15.123:0/3399614409 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2c300c7460 0x7f2c30150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 >> 172.21.15.123:0/3399614409 conn(0x7f2c300c9c90 msgr2=0x7f2c30155c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:00.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 shutdown_connections 2023-10-06T04:33:00.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.525+0000 7f2c38181700 1 -- 172.21.15.123:0/3399614409 wait complete. 2023-10-06T04:33:00.528 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 26 2023-10-06T04:33:00.538 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:33:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.653+0000 7f4dce4ee700 1 Processor -- start 2023-10-06T04:33:00.655 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.653+0000 7f4dce4ee700 1 -- start start 2023-10-06T04:33:00.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dce4ee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4dc80c9570 0x7f4dc80a9940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dce4ee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80a9e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dce4ee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4dc806a750 con 0x7f4dc8067660 2023-10-06T04:33:00.657 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dce4ee700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4dc806aa30 con 0x7f4dc80c9570 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dce4ee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc806ad10 con 0x7f4dc80ca5b0 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dc77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4dc80c9570 0x7f4dc80a9940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dc77fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f4dc80c9570 0x7f4dc80a9940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53190/0 (socket says 172.21.15.123:53190) 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dc77fe700 1 -- 172.21.15.123:0/587799771 learned_addr learned my addr 172.21.15.123:0/587799771 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.654+0000 7f4dcca8b700 1 --2- 172.21.15.123:0/587799771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80a9e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc77fe700 1 -- 172.21.15.123:0/587799771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 msgr2=0x7f4dc80a9e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc77fe700 1 --2- 172.21.15.123:0/587799771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80a9e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.658 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc77fe700 1 -- 172.21.15.123:0/587799771 >> v1:172.21.15.177:6789/0 conn(0x7f4dc8067660 legacy=0x7f4dc8067a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc77fe700 1 -- 172.21.15.123:0/587799771 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dc806aff0 con 0x7f4dc80c9570 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc77fe700 1 --2- 172.21.15.123:0/587799771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc80a9940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0099b0 tx=0x7f4dbc009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f049bb2d3048cc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc6ffd700 1 -- 172.21.15.123:0/587799771 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc009ea0 con 0x7f4dc80c9570 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc6ffd700 1 -- 172.21.15.123:0/587799771 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc00b860 con 0x7f4dc80c9570 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dc6ffd700 1 -- 172.21.15.123:0/587799771 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00bc00 con 0x7f4dc80c9570 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dce4ee700 1 -- 172.21.15.123:0/587799771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 msgr2=0x7f4dc80a9940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.655+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/587799771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc80a9940 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f4dbc0099b0 tx=0x7f4dbc009250 comp rx=0 tx=0).stop 2023-10-06T04:33:00.659 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.656+0000 7f4dce4ee700 1 -- 172.21.15.123:0/587799771 shutdown_connections 2023-10-06T04:33:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.656+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/587799771 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80a9e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.656+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/587799771 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc80a9940 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.656+0000 7f4dce4ee700 1 -- 172.21.15.123:0/587799771 >> 172.21.15.123:0/587799771 conn(0x7f4dc80cda80 msgr2=0x7f4dc80c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.656+0000 7f4dce4ee700 1 -- 172.21.15.123:0/587799771 shutdown_connections 2023-10-06T04:33:00.660 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.656+0000 7f4dce4ee700 1 -- 172.21.15.123:0/587799771 wait complete. 2023-10-06T04:33:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.657+0000 7f4dce4ee700 1 Processor -- start 2023-10-06T04:33:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.657+0000 7f4dce4ee700 1 -- start start 2023-10-06T04:33:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dc8067660 0x7f4dc80ad960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc815cab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.663 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80aefe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80aeb50 con 0x7f4dc80c9570 2023-10-06T04:33:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80adf40 con 0x7f4dc8067660 2023-10-06T04:33:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4dc80ae220 con 0x7f4dc80ca5b0 2023-10-06T04:33:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc815cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc815cab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53196/0 (socket says 172.21.15.123:53196) 2023-10-06T04:33:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 -- 172.21.15.123:0/3906250893 learned_addr learned my addr 172.21.15.123:0/3906250893 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:00.664 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dcca8b700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80aefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc7fff700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dc8067660 0x7f4dc80ad960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 -- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 msgr2=0x7f4dc80aefe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80aefe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 -- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dc8067660 msgr2=0x7f4dc80ad960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dc8067660 0x7f4dc80ad960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 -- 172.21.15.123:0/3906250893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4dbc0094f0 con 0x7f4dc80c9570 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dc77fe700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc815cab0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806f7f0 tx=0x7f4dbc017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:00.665 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4daffff700 1 -- 172.21.15.123:0/3906250893 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc00bbe0 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4daffff700 1 -- 172.21.15.123:0/3906250893 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4dbc011420 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4daffff700 1 -- 172.21.15.123:0/3906250893 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4dbc0117c0 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4dc80ae500 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.658+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4dc815cff0 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.659+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4dc815efe0 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.660+0000 7f4daffff700 1 -- 172.21.15.123:0/3906250893 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4dbc011bf0 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.660+0000 7f4daffff700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4db4066f30 0x7f4db40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.660+0000 7f4daffff700 1 -- 172.21.15.123:0/3906250893 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f4dbc086330 con 0x7f4dc80c9570 2023-10-06T04:33:00.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.661+0000 7f4dc7fff700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4db4066f30 0x7f4db40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:00.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.661+0000 7f4dc7fff700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4db4066f30 0x7f4db40693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4db80037c0 tx=0x7f4db8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:00.667 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.664+0000 7f4daffff700 1 -- 172.21.15.123:0/3906250893 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f4dbc0532a0 con 0x7f4dc80c9570 2023-10-06T04:33:00.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.887+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4dc80cde80 con 0x7f4dc80c9570 2023-10-06T04:33:00.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.887+0000 7f4daffff700 1 -- 172.21.15.123:0/3906250893 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+5671 (secure 0 0 0) 0x7f4dbc0568c0 con 0x7f4dc80c9570 2023-10-06T04:33:00.889 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:00.889 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":26,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":18,"flags":19,"flags_state":{"joinable":false,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:23.822729+0000","modified":"2023-10-06T04:32:33.579847+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":35,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{},"failed":[0],"damaged":[],"stopped":[],"info":{},"data_pools":[6],"metadata_pool":5,"enabled":true,"fs_name":"cephfs","balancer":"","standby_count_wanted":0},"id":2},{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:33:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.889+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4db4066f30 msgr2=0x7f4db40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.889+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4db4066f30 0x7f4db40693e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f4db80037c0 tx=0x7f4db8009040 comp rx=0 tx=0).stop 2023-10-06T04:33:00.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.889+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 msgr2=0x7f4dc815cab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.889+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc815cab0 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7f4dc806f7f0 tx=0x7f4dbc017000 comp rx=0 tx=0).stop 2023-10-06T04:33:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.889+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 shutdown_connections 2023-10-06T04:33:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.889+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4db4066f30 0x7f4db40693e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.890+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4dc80ca5b0 0x7f4dc80aefe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.890+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4dc8067660 0x7f4dc80ad960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.890+0000 7f4dce4ee700 1 --2- 172.21.15.123:0/3906250893 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4dc80c9570 0x7f4dc815cab0 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:00.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.890+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 >> 172.21.15.123:0/3906250893 conn(0x7f4dc80cda80 msgr2=0x7f4dc80cc540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:00.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.890+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 shutdown_connections 2023-10-06T04:33:00.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:00.890+0000 7f4dce4ee700 1 -- 172.21.15.123:0/3906250893 wait complete. 2023-10-06T04:33:00.893 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 26 2023-10-06T04:33:00.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:33:01.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.018+0000 7fa83b6c4700 1 Processor -- start 2023-10-06T04:33:01.020 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.019+0000 7fa83b6c4700 1 -- start start 2023-10-06T04:33:01.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.019+0000 7fa83b6c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa8340c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.022 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa83b6c4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa83406e960 0x7fa8340ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa83b6c4700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8340c8580 con 0x7fa83406f080 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa83b6c4700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8340c8860 con 0x7fa83406e960 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa83b6c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340c8b40 con 0x7fa8341560c0 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa839c61700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa83406e960 0x7fa8340ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa838c5f700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa83406f080 0x7fa83406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60500/0 (socket says 172.21.15.123:60500) 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa838c5f700 1 -- 172.21.15.123:0/3003675015 learned_addr learned my addr 172.21.15.123:0/3003675015 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa839460700 1 --2- 172.21.15.123:0/3003675015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa8340c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa839c61700 1 -- 172.21.15.123:0/3003675015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 msgr2=0x7fa8340c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:01.023 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa839c61700 1 --2- 172.21.15.123:0/3003675015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa8340c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa839c61700 1 -- 172.21.15.123:0/3003675015 >> v1:172.21.15.123:6789/0 conn(0x7fa83406f080 legacy=0x7fa83406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa839c61700 1 -- 172.21.15.123:0/3003675015 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8340c8e20 con 0x7fa83406e960 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa82bfff700 1 -- 172.21.15.123:0/3003675015 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2804180939 0 0) 0x7fa8340c8580 con 0x7fa83406f080 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa839c61700 1 --2- 172.21.15.123:0/3003675015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406e960 0x7fa8340ada60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00d0d0 tx=0x7fa82c00b990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2d0acd878c784b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa82bfff700 1 -- 172.21.15.123:0/3003675015 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c010070 con 0x7fa83406e960 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa82bfff700 1 -- 172.21.15.123:0/3003675015 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa82c00ddf0 con 0x7fa83406e960 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.020+0000 7fa82bfff700 1 -- 172.21.15.123:0/3003675015 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa82c00b320 con 0x7fa83406e960 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 -- 172.21.15.123:0/3003675015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406e960 msgr2=0x7fa8340ada60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/3003675015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406e960 0x7fa8340ada60 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fa82c00d0d0 tx=0x7fa82c00b990 comp rx=0 tx=0).stop 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 -- 172.21.15.123:0/3003675015 shutdown_connections 2023-10-06T04:33:01.024 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/3003675015 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406e960 0x7fa8340ada60 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/3003675015 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa8340c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.025 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 -- 172.21.15.123:0/3003675015 >> 172.21.15.123:0/3003675015 conn(0x7fa8340cda80 msgr2=0x7fa8340c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:01.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 -- 172.21.15.123:0/3003675015 shutdown_connections 2023-10-06T04:33:01.026 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.021+0000 7fa83b6c4700 1 -- 172.21.15.123:0/3003675015 wait complete. 2023-10-06T04:33:01.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.022+0000 7fa83b6c4700 1 Processor -- start 2023-10-06T04:33:01.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.022+0000 7fa83b6c4700 1 -- start start 2023-10-06T04:33:01.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.023+0000 7fa83b6c4700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa83406e960 0x7fa834153790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.023+0000 7fa83b6c4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406f080 0x7fa8340b0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.029 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa83b6c4700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa834154280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa83b6c4700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa834157c90 con 0x7fa83406f080 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa83b6c4700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8341573a0 con 0x7fa83406e960 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa83b6c4700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8340b1910 con 0x7fa8341560c0 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406f080 0x7fa8340b0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406f080 0x7fa8340b0fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53222/0 (socket says 172.21.15.123:53222) 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 -- 172.21.15.123:0/357744401 learned_addr learned my addr 172.21.15.123:0/357744401 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa839c61700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa834154280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa839460700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa83406e960 0x7fa834153790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 -- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 msgr2=0x7fa834154280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa834154280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.030 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 -- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa83406e960 msgr2=0x7fa834153790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa83406e960 0x7fa834153790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 -- 172.21.15.123:0/357744401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa82c00bc30 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa839460700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa83406e960 0x7fa834153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa838c5f700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406f080 0x7fa8340b0fc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c7e0 tx=0x7fa830007df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa8297fa700 1 -- 172.21.15.123:0/357744401 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830005b30 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa8297fa700 1 -- 172.21.15.123:0/357744401 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa830005c90 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa834157640 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa8297fa700 1 -- 172.21.15.123:0/357744401 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa830007300 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.024+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa834153cd0 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.025+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa834157e00 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.026+0000 7fa8297fa700 1 -- 172.21.15.123:0/357744401 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa8300076b0 con 0x7fa83406f080 2023-10-06T04:33:01.031 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.026+0000 7fa8297fa700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa818066f30 0x7fa8180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.026+0000 7fa8297fa700 1 -- 172.21.15.123:0/357744401 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fa830083e20 con 0x7fa83406f080 2023-10-06T04:33:01.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.026+0000 7fa839460700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa818066f30 0x7fa8180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.027+0000 7fa839460700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa818066f30 0x7fa8180693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:01.032 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.030+0000 7fa8297fa700 1 -- 172.21.15.123:0/357744401 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa830050d10 con 0x7fa83406f080 2023-10-06T04:33:01.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.212+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8340cde80 con 0x7fa83406f080 2023-10-06T04:33:01.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.213+0000 7fa8297fa700 1 -- 172.21.15.123:0/357744401 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v42) v1 ==== 74+0+18770 (secure 0 0 0) 0x7fa830017090 con 0x7fa83406f080 2023-10-06T04:33:01.215 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:01.215 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":42,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:32:38.586940+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":5,"pool_name":"cephfs_metadata","create_time":"2023-10-06T04:32:21.900854+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"33","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:33:01.216 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa818066f30 msgr2=0x7fa8180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa818066f30 0x7fa8180693e0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fa824006fd0 tx=0x7fa824008040 comp rx=0 tx=0).stop 2023-10-06T04:33:01.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406f080 msgr2=0x7fa8340b0fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406f080 0x7fa8340b0fc0 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fa83000c7e0 tx=0x7fa830007df0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 shutdown_connections 2023-10-06T04:33:01.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa818066f30 0x7fa8180693e0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa8341560c0 0x7fa834154280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.217 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa83406e960 0x7fa834153790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 --2- 172.21.15.123:0/357744401 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa83406f080 0x7fa8340b0fc0 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 >> 172.21.15.123:0/357744401 conn(0x7fa8340cda80 msgr2=0x7fa8341554c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:01.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 shutdown_connections 2023-10-06T04:33:01.218 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.215+0000 7fa83b6c4700 1 -- 172.21.15.123:0/357744401 wait complete. 2023-10-06T04:33:01.229 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-10-06T04:33:01.350 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.349+0000 7fb243e97700 1 Processor -- start 2023-10-06T04:33:01.351 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.349+0000 7fb243e97700 1 -- start start 2023-10-06T04:33:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb243e97700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb23c1594a0 0x7fb23c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb243e97700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb23c06e450 0x7fb23c06e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb243e97700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb23c0cad40 con 0x7fb23c068670 2023-10-06T04:33:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb243e97700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0ca6a0 con 0x7fb23c1594a0 2023-10-06T04:33:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb243e97700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb23c0cb550 con 0x7fb23c06e450 2023-10-06T04:33:01.354 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb241c33700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb23c1594a0 0x7fb23c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb241c33700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb23c1594a0 0x7fb23c0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53236/0 (socket says 172.21.15.123:53236) 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb241c33700 1 -- 172.21.15.123:0/2888999153 learned_addr learned my addr 172.21.15.123:0/2888999153 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.350+0000 7fb241432700 1 --2- 172.21.15.123:0/2888999153 >> v2:172.21.15.177:3300/0 conn(0x7fb23c06e450 0x7fb23c06e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb241c33700 1 -- 172.21.15.123:0/2888999153 >> v2:172.21.15.177:3300/0 conn(0x7fb23c06e450 msgr2=0x7fb23c06e8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb241c33700 1 --2- 172.21.15.123:0/2888999153 >> v2:172.21.15.177:3300/0 conn(0x7fb23c06e450 0x7fb23c06e8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb241c33700 1 -- 172.21.15.123:0/2888999153 >> v1:172.21.15.177:6789/0 conn(0x7fb23c068670 legacy=0x7fb23c068a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb241c33700 1 -- 172.21.15.123:0/2888999153 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb23c06a920 con 0x7fb23c1594a0 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb241432700 1 --2- 172.21.15.123:0/2888999153 >> v2:172.21.15.177:3300/0 conn(0x7fb23c06e450 0x7fb23c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb241c33700 1 --2- 172.21.15.123:0/2888999153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c1594a0 0x7fb23c0c6dd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b2970 tx=0x7fb22c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74902ac67791b0b0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:01.355 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb240c31700 1 -- 172.21.15.123:0/2888999153 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00b860 con 0x7fb23c1594a0 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb240c31700 1 -- 172.21.15.123:0/2888999153 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c00b9c0 con 0x7fb23c1594a0 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb240c31700 1 -- 172.21.15.123:0/2888999153 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c00bd60 con 0x7fb23c1594a0 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb243e97700 1 -- 172.21.15.123:0/2888999153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c1594a0 msgr2=0x7fb23c0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb243e97700 1 --2- 172.21.15.123:0/2888999153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c1594a0 0x7fb23c0c6dd0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b2970 tx=0x7fb22c009250 comp rx=0 tx=0).stop 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb243e97700 1 -- 172.21.15.123:0/2888999153 shutdown_connections 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb243e97700 1 --2- 172.21.15.123:0/2888999153 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c1594a0 0x7fb23c0c6dd0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb243e97700 1 --2- 172.21.15.123:0/2888999153 >> v2:172.21.15.177:3300/0 conn(0x7fb23c06e450 0x7fb23c06e8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.351+0000 7fb243e97700 1 -- 172.21.15.123:0/2888999153 >> 172.21.15.123:0/2888999153 conn(0x7fb23c0c9d00 msgr2=0x7fb23c06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.352+0000 7fb243e97700 1 -- 172.21.15.123:0/2888999153 shutdown_connections 2023-10-06T04:33:01.356 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.352+0000 7fb243e97700 1 -- 172.21.15.123:0/2888999153 wait complete. 2023-10-06T04:33:01.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 Processor -- start 2023-10-06T04:33:01.357 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 -- start start 2023-10-06T04:33:01.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb23c068670 0x7fb23c15a4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c06e450 0x7fb23c15acc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23c1594a0 0x7fb23c1515c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c0af7d0 con 0x7fb23c06e450 2023-10-06T04:33:01.359 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c151b00 con 0x7fb23c068670 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb243e97700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb23c16a230 con 0x7fb23c1594a0 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb241432700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c06e450 0x7fb23c15acc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb241432700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c06e450 0x7fb23c15acc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53246/0 (socket says 172.21.15.123:53246) 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb241432700 1 -- 172.21.15.123:0/2937261330 learned_addr learned my addr 172.21.15.123:0/2937261330 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb241c33700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb23c068670 0x7fb23c15a4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.353+0000 7fb242434700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23c1594a0 0x7fb23c1515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb241432700 1 -- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23c1594a0 msgr2=0x7fb23c1515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb241432700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23c1594a0 0x7fb23c1515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb241432700 1 -- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb23c068670 msgr2=0x7fb23c15a4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb241432700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb23c068670 0x7fb23c15a4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:01.360 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb241432700 1 -- 172.21.15.123:0/2937261330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb22c0094f0 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb241432700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c06e450 0x7fb23c15acc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb23800e900 tx=0x7fb238008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb231ffb700 1 -- 172.21.15.123:0/2937261330 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238013410 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb231ffb700 1 -- 172.21.15.123:0/2937261330 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb238013570 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb23c16a510 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb231ffb700 1 -- 172.21.15.123:0/2937261330 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb238013910 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.354+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23c16aad0 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.355+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb23c0af980 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.355+0000 7fb231ffb700 1 -- 172.21.15.123:0/2937261330 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb238013d00 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.356+0000 7fb231ffb700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb220066f30 0x7fb2200693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.356+0000 7fb231ffb700 1 -- 172.21.15.123:0/2937261330 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7fb238084690 con 0x7fb23c06e450 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.356+0000 7fb241c33700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb220066f30 0x7fb2200693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:01.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.356+0000 7fb241c33700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb220066f30 0x7fb2200693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b2970 tx=0x7fb22c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:01.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.360+0000 7fb231ffb700 1 -- 172.21.15.123:0/2937261330 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb23c0af980 con 0x7fb23c06e450 2023-10-06T04:33:01.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:01.587+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fb23c0af980 con 0x7fb23c06e450 2023-10-06T04:33:02.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.297+0000 7fb231ffb700 1 -- 172.21.15.123:0/2937261330 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v27) v1 ==== 161+0+0 (secure 0 0 0) 0x7fb23c0af980 con 0x7fb23c06e450 2023-10-06T04:33:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb220066f30 msgr2=0x7fb2200693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb220066f30 0x7fb2200693e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fb23c0b2970 tx=0x7fb22c017000 comp rx=0 tx=0).stop 2023-10-06T04:33:02.301 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c06e450 msgr2=0x7fb23c15acc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c06e450 0x7fb23c15acc0 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb23800e900 tx=0x7fb238008040 comp rx=0 tx=0).stop 2023-10-06T04:33:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 shutdown_connections 2023-10-06T04:33:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb220066f30 0x7fb2200693e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb23c1594a0 0x7fb23c1515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb23c068670 0x7fb23c15a4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 --2- 172.21.15.123:0/2937261330 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb23c06e450 0x7fb23c15acc0 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.302 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 >> 172.21.15.123:0/2937261330 conn(0x7fb23c0c9d00 msgr2=0x7fb23c1564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:02.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 shutdown_connections 2023-10-06T04:33:02.303 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.300+0000 7fb243e97700 1 -- 172.21.15.123:0/2937261330 wait complete. 2023-10-06T04:33:02.303 INFO:teuthology.orchestra.run.smithi123.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:33:02.315 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-10-06T04:33:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.434+0000 7f92dad8d700 1 Processor -- start 2023-10-06T04:33:02.436 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.434+0000 7f92dad8d700 1 -- start start 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.435+0000 7f92dad8d700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f92d41595e0 0x7f92d40c1d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.435+0000 7f92dad8d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d40c0d70 con 0x7f92d40c22c0 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.435+0000 7f92dad8d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92d40c18f0 con 0x7f92d4159210 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.435+0000 7f92dad8d700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92d40c0fc0 con 0x7f92d41595e0 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.435+0000 7f92d932a700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f92d40c22c0 0x7f92d40a98a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60506/0 (socket says 172.21.15.123:60506) 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.435+0000 7f92d932a700 1 -- 172.21.15.123:0/3881345602 learned_addr learned my addr 172.21.15.123:0/3881345602 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014473512 0 0) 0x7f92d40c0d70 con 0x7f92d40c22c0 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d3fff700 1 --2- 172.21.15.123:0/3881345602 >> v2:172.21.15.177:3300/0 conn(0x7f92d41595e0 0x7f92d40c1d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.440 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92c4003570 con 0x7f92d40c22c0 2023-10-06T04:33:02.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1306234312 0 0) 0x7f92c4003570 con 0x7f92d40c22c0 2023-10-06T04:33:02.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 >> v2:172.21.15.177:3300/0 conn(0x7f92d41595e0 msgr2=0x7f92d40c1d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:02.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 --2- 172.21.15.123:0/3881345602 >> v2:172.21.15.177:3300/0 conn(0x7f92d41595e0 0x7f92d40c1d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 >> v1:172.21.15.177:6789/0 conn(0x7f92d4159210 legacy=0x7f92d40c7000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:02.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40c12a0 con 0x7f92d40c22c0 2023-10-06T04:33:02.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f92cc0017f0 con 0x7f92d40c22c0 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 >> v1:172.21.15.123:6789/0 conn(0x7f92d40c22c0 legacy=0x7f92d40a98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4003a60 0x7f92c4003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4004810 0x7f92c4004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92c40051c0 0x7f92c400ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d40c0fc0 con 0x7f92c4003a60 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92c400c160 con 0x7f92c4004810 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92c400c320 con 0x7f92c40051c0 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d8b29700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4003a60 0x7f92c4003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d3fff700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4004810 0x7f92c4004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d932a700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92c40051c0 0x7f92c400ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d8b29700 1 -- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92c40051c0 msgr2=0x7f92c400ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d8b29700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92c40051c0 0x7f92c400ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d8b29700 1 -- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4004810 msgr2=0x7f92c4004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d8b29700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4004810 0x7f92c4004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d8b29700 1 -- 172.21.15.123:0/3881345602 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92c4003570 con 0x7f92c4003a60 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d932a700 1 -- 172.21.15.123:0/3881345602 reap_dead start 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.436+0000 7f92d8b29700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4003a60 0x7f92c4003e70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f92c80018b0 tx=0x7f92c8007d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32ce1947a6839c94 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8012420 con 0x7f92c4003a60 2023-10-06T04:33:02.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c80125a0 con 0x7f92c4003a60 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92d37fe700 1 -- 172.21.15.123:0/3881345602 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c8012960 con 0x7f92c4003a60 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92dad8d700 1 -- 172.21.15.123:0/3881345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4003a60 msgr2=0x7f92c4003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4003a60 0x7f92c4003e70 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f92c80018b0 tx=0x7f92c8007d30 comp rx=0 tx=0).stop 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92dad8d700 1 -- 172.21.15.123:0/3881345602 shutdown_connections 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3881345602 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92c4003a60 0x7f92c4003e70 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92dad8d700 1 -- 172.21.15.123:0/3881345602 >> 172.21.15.123:0/3881345602 conn(0x7f92d40cbf20 msgr2=0x7f92d40cc320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92dad8d700 1 -- 172.21.15.123:0/3881345602 shutdown_connections 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.437+0000 7f92dad8d700 1 -- 172.21.15.123:0/3881345602 wait complete. 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.438+0000 7f92dad8d700 1 Processor -- start 2023-10-06T04:33:02.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.438+0000 7f92dad8d700 1 -- start start 2023-10-06T04:33:02.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92dad8d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4003a60 0x7f92d40abda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92dad8d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d413fd20 0x7f92d416a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92dad8d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92d415abc0 0x7f92d415b030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92dad8d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d415b570 con 0x7f92d413fd20 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92dad8d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92d40b0340 con 0x7f92c4003a60 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92dad8d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92d416a7f0 con 0x7f92d415abc0 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92d3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d413fd20 0x7f92d416a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92d3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d413fd20 0x7f92d416a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53260/0 (socket says 172.21.15.123:53260) 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92d3fff700 1 -- 172.21.15.123:0/3898553355 learned_addr learned my addr 172.21.15.123:0/3898553355 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92d932a700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92d415abc0 0x7f92d415b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.439+0000 7f92d8b29700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4003a60 0x7f92d40abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d3fff700 1 -- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92d415abc0 msgr2=0x7f92d415b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d3fff700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92d415abc0 0x7f92d415b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d3fff700 1 -- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4003a60 msgr2=0x7f92d40abda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d3fff700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4003a60 0x7f92d40abda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d3fff700 1 -- 172.21.15.123:0/3898553355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d40ac380 con 0x7f92d413fd20 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d3fff700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d413fd20 0x7f92d416a2b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f92c0011b40 tx=0x7f92c00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d0ff9700 1 -- 172.21.15.123:0/3898553355 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c000f940 con 0x7f92d413fd20 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d0ff9700 1 -- 172.21.15.123:0/3898553355 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92c000faa0 con 0x7f92d413fd20 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d40ac660 con 0x7f92d413fd20 2023-10-06T04:33:02.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92d0ff9700 1 -- 172.21.15.123:0/3898553355 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92c000fe20 con 0x7f92d413fd20 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.440+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92d40acc20 con 0x7f92d413fd20 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.441+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92d40b0110 con 0x7f92d413fd20 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.441+0000 7f92d0ff9700 1 -- 172.21.15.123:0/3898553355 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f92c0014650 con 0x7f92d413fd20 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.442+0000 7f92d0ff9700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92b8066f30 0x7f92b80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.442+0000 7f92d8b29700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92b8066f30 0x7f92b80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.442+0000 7f92d0ff9700 1 -- 172.21.15.123:0/3898553355 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f92c0018030 con 0x7f92d413fd20 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.442+0000 7f92d8b29700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92b8066f30 0x7f92b80693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f92d41403e0 tx=0x7f92c8015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:02.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.446+0000 7f92d0ff9700 1 -- 172.21.15.123:0/3898553355 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f92d40b0110 con 0x7f92d413fd20 2023-10-06T04:33:02.672 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:02.670+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f92d40b0110 con 0x7f92d413fd20 2023-10-06T04:33:03.306 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.305+0000 7f92d0ff9700 1 -- 172.21.15.123:0/3898553355 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v28) v1 ==== 104+0+0 (secure 0 0 0) 0x7f92d40b0110 con 0x7f92d413fd20 2023-10-06T04:33:03.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92b8066f30 msgr2=0x7f92b80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92b8066f30 0x7f92b80693e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f92d41403e0 tx=0x7f92c8015000 comp rx=0 tx=0).stop 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d413fd20 msgr2=0x7f92d416a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d413fd20 0x7f92d416a2b0 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f92c0011b40 tx=0x7f92c00091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 shutdown_connections 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f92b8066f30 0x7f92b80693e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f92d415abc0 0x7f92d415b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f92c4003a60 0x7f92d40abda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 --2- 172.21.15.123:0/3898553355 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f92d413fd20 0x7f92d416a2b0 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 >> 172.21.15.123:0/3898553355 conn(0x7f92d40cbf20 msgr2=0x7f92d4157a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:03.310 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 shutdown_connections 2023-10-06T04:33:03.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.308+0000 7f92dad8d700 1 -- 172.21.15.123:0/3898553355 wait complete. 2023-10-06T04:33:03.324 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-10-06T04:33:03.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.449+0000 7f8bba4e6700 1 Processor -- start 2023-10-06T04:33:03.451 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.449+0000 7f8bba4e6700 1 -- start start 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bba4e6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8bb4068d80 0x7f8bb40a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bba4e6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8bb41594e0 0x7f8bb40a9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bba4e6700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8bb40aa7b0 con 0x7f8bb4159110 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bba4e6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40af130 con 0x7f8bb4068d80 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bba4e6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bb40ab150 con 0x7f8bb41594e0 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb37fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8bb4068d80 0x7f8bb40a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb37fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8bb4068d80 0x7f8bb40a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53262/0 (socket says 172.21.15.123:53262) 2023-10-06T04:33:03.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb37fe700 1 -- 172.21.15.123:0/3487881881 learned_addr learned my addr 172.21.15.123:0/3487881881 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb8a83700 1 --2- 172.21.15.123:0/3487881881 >> v2:172.21.15.177:3300/0 conn(0x7f8bb41594e0 0x7f8bb40a9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb37fe700 1 -- 172.21.15.123:0/3487881881 >> v2:172.21.15.177:3300/0 conn(0x7f8bb41594e0 msgr2=0x7f8bb40a9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb37fe700 1 --2- 172.21.15.123:0/3487881881 >> v2:172.21.15.177:3300/0 conn(0x7f8bb41594e0 0x7f8bb40a9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb37fe700 1 -- 172.21.15.123:0/3487881881 >> v1:172.21.15.177:6789/0 conn(0x7f8bb4159110 legacy=0x7f8bb4068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.450+0000 7f8bb37fe700 1 -- 172.21.15.123:0/3487881881 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bb40aa4d0 con 0x7f8bb4068d80 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bb37fe700 1 --2- 172.21.15.123:0/3487881881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb40a98a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40adf00 tx=0x7f8ba4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=521bc134efe55324 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bb2ffd700 1 -- 172.21.15.123:0/3487881881 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400b860 con 0x7f8bb4068d80 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bb2ffd700 1 -- 172.21.15.123:0/3487881881 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba400b9c0 con 0x7f8bb4068d80 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bb2ffd700 1 -- 172.21.15.123:0/3487881881 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba400bd80 con 0x7f8bb4068d80 2023-10-06T04:33:03.454 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3487881881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 msgr2=0x7f8bb40a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:03.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3487881881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb40a98a0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40adf00 tx=0x7f8ba4009250 comp rx=0 tx=0).stop 2023-10-06T04:33:03.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3487881881 shutdown_connections 2023-10-06T04:33:03.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3487881881 >> v2:172.21.15.177:3300/0 conn(0x7f8bb41594e0 0x7f8bb40a9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3487881881 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb40a98a0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3487881881 >> 172.21.15.123:0/3487881881 conn(0x7f8bb406dac0 msgr2=0x7f8bb406dec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:03.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3487881881 shutdown_connections 2023-10-06T04:33:03.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.451+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3487881881 wait complete. 2023-10-06T04:33:03.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.452+0000 7f8bba4e6700 1 Processor -- start 2023-10-06T04:33:03.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bba4e6700 1 -- start start 2023-10-06T04:33:03.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bba4e6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb415e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:03.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bba4e6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4159110 0x7f8bb415b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:03.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bba4e6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bb41594e0 0x7f8bb4150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:03.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bba4e6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb415ad20 con 0x7f8bb4068d80 2023-10-06T04:33:03.458 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bba4e6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb415a760 con 0x7f8bb4159110 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bba4e6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bb415aa40 con 0x7f8bb41594e0 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bb3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb415e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bb3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb415e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53268/0 (socket says 172.21.15.123:53268) 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.453+0000 7f8bb3fff700 1 -- 172.21.15.123:0/3063129796 learned_addr learned my addr 172.21.15.123:0/3063129796 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb37fe700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4159110 0x7f8bb415b2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb8a83700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bb41594e0 0x7f8bb4150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb3fff700 1 -- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bb41594e0 msgr2=0x7f8bb4150500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb3fff700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bb41594e0 0x7f8bb4150500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb3fff700 1 -- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4159110 msgr2=0x7f8bb415b2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb3fff700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4159110 0x7f8bb415b2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:03.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb3fff700 1 -- 172.21.15.123:0/3063129796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ba40094f0 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bb3fff700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb415e3c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c4e0 tx=0x7f8ba800c7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8b9bfff700 1 -- 172.21.15.123:0/3063129796 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8011590 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8b9bfff700 1 -- 172.21.15.123:0/3063129796 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ba8011710 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bb4155dd0 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8b9bfff700 1 -- 172.21.15.123:0/3063129796 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ba8011ad0 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.454+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bb4151420 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.455+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bb41562f0 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.455+0000 7f8b9bfff700 1 -- 172.21.15.123:0/3063129796 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8ba801b3b0 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.456+0000 7f8b9bfff700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8ba0066f30 0x7f8ba00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.456+0000 7f8b9bfff700 1 -- 172.21.15.123:0/3063129796 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6597+0+0 (secure 0 0 0) 0x7f8ba8083cb0 con 0x7f8bb4068d80 2023-10-06T04:33:03.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.456+0000 7f8bb37fe700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8ba0066f30 0x7f8ba00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:03.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.456+0000 7f8bb37fe700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8ba0066f30 0x7f8ba00693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40adf00 tx=0x7f8ba4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:03.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.460+0000 7f8b9bfff700 1 -- 172.21.15.123:0/3063129796 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8bb41562f0 con 0x7f8bb4068d80 2023-10-06T04:33:03.549 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:03.551 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:03.652 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:03.651+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8bb41562f0 con 0x7f8bb4068d80 2023-10-06T04:33:04.313 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.312+0000 7f8b9bfff700 1 -- 172.21.15.123:0/3063129796 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8ba8017070 con 0x7f8bb4068d80 2023-10-06T04:33:04.317 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8ba0066f30 msgr2=0x7f8ba00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8ba0066f30 0x7f8ba00693e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f8bb40adf00 tx=0x7f8ba4017000 comp rx=0 tx=0).stop 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 msgr2=0x7f8bb415e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb415e3c0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f8ba800c4e0 tx=0x7f8ba800c7f0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 shutdown_connections 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8ba0066f30 0x7f8ba00693e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8bb4068d80 0x7f8bb415e3c0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8bb41594e0 0x7f8bb4150500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 --2- 172.21.15.123:0/3063129796 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8bb4159110 0x7f8bb415b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 >> 172.21.15.123:0/3063129796 conn(0x7f8bb406dac0 msgr2=0x7f8bb40cbeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:04.318 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 shutdown_connections 2023-10-06T04:33:04.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.316+0000 7f8bba4e6700 1 -- 172.21.15.123:0/3063129796 wait complete. 2023-10-06T04:33:04.319 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_metadata' removed 2023-10-06T04:33:04.330 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:33:04.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.491+0000 7f8b62694700 1 Processor -- start 2023-10-06T04:33:04.493 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.491+0000 7f8b62694700 1 -- start start 2023-10-06T04:33:04.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b62694700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 0x7f8b5c0ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b62694700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8b5c0ae6d0 0x7f8b5c15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.495 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b62694700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b5c0ad350 con 0x7f8b5c0acd90 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b62694700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b5c157ba0 con 0x7f8b5c0ae6d0 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b62694700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c0ad630 con 0x7f8b5c1573e0 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b60c31700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8b5c0ae6d0 0x7f8b5c15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b60c31700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f8b5c0ae6d0 0x7f8b5c15fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53282/0 (socket says 172.21.15.123:53282) 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b60c31700 1 -- 172.21.15.123:0/3478710437 learned_addr learned my addr 172.21.15.123:0/3478710437 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.492+0000 7f8b5b7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8b5c0acd90 0x7f8b5c0adfc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60508/0 (socket says 172.21.15.123:60508) 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b5bfff700 1 --2- 172.21.15.123:0/3478710437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 0x7f8b5c0ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b60c31700 1 -- 172.21.15.123:0/3478710437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 msgr2=0x7f8b5c0ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b60c31700 1 --2- 172.21.15.123:0/3478710437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 0x7f8b5c0ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b60c31700 1 -- 172.21.15.123:0/3478710437 >> v1:172.21.15.123:6789/0 conn(0x7f8b5c0acd90 legacy=0x7f8b5c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.496 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b60c31700 1 -- 172.21.15.123:0/3478710437 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c0ad910 con 0x7f8b5c0ae6d0 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b5affd700 1 -- 172.21.15.123:0/3478710437 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1540836788 0 0) 0x7f8b5c0ad350 con 0x7f8b5c0acd90 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b60c31700 1 --2- 172.21.15.123:0/3478710437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0ae6d0 0x7f8b5c15fcd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400d1a0 tx=0x7f8b5400ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e80316b0a59c385 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b5affd700 1 -- 172.21.15.123:0/3478710437 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b54010070 con 0x7f8b5c0ae6d0 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b5affd700 1 -- 172.21.15.123:0/3478710437 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5400b0d0 con 0x7f8b5c0ae6d0 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.493+0000 7f8b5affd700 1 -- 172.21.15.123:0/3478710437 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5400b490 con 0x7f8b5c0ae6d0 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 -- 172.21.15.123:0/3478710437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0ae6d0 msgr2=0x7f8b5c15fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 --2- 172.21.15.123:0/3478710437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0ae6d0 0x7f8b5c15fcd0 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f8b5400d1a0 tx=0x7f8b5400ba50 comp rx=0 tx=0).stop 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 -- 172.21.15.123:0/3478710437 shutdown_connections 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 --2- 172.21.15.123:0/3478710437 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0ae6d0 0x7f8b5c15fcd0 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 --2- 172.21.15.123:0/3478710437 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 0x7f8b5c0ac850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.497 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 -- 172.21.15.123:0/3478710437 >> 172.21.15.123:0/3478710437 conn(0x7f8b5c0cdb10 msgr2=0x7f8b5c0abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:04.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 -- 172.21.15.123:0/3478710437 shutdown_connections 2023-10-06T04:33:04.498 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.494+0000 7f8b62694700 1 -- 172.21.15.123:0/3478710437 wait complete. 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.495+0000 7f8b62694700 1 Processor -- start 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.495+0000 7f8b62694700 1 -- start start 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0acd90 0x7f8b5c142f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b5c0ae6d0 0x7f8b5c06e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 0x7f8b5c06e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c1434c0 con 0x7f8b5c0acd90 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c096ad0 con 0x7f8b5c0ae6d0 2023-10-06T04:33:04.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5c096db0 con 0x7f8b5c1573e0 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0acd90 0x7f8b5c142f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0acd90 0x7f8b5c142f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53288/0 (socket says 172.21.15.123:53288) 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 -- 172.21.15.123:0/3232705870 learned_addr learned my addr 172.21.15.123:0/3232705870 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 -- 172.21.15.123:0/3232705870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 msgr2=0x7f8b5c06e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 --2- 172.21.15.123:0/3232705870 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8b5c1573e0 0x7f8b5c06e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 -- 172.21.15.123:0/3232705870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b5c0ae6d0 msgr2=0x7f8b5c06e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 --2- 172.21.15.123:0/3232705870 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8b5c0ae6d0 0x7f8b5c06e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 -- 172.21.15.123:0/3232705870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5400bcf0 con 0x7f8b5c0acd90 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b5bfff700 1 --2- 172.21.15.123:0/3232705870 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8b5c0acd90 0x7f8b5c142f80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f8b4c00d890 tx=0x7f8b4c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b43fff700 1 -- 172.21.15.123:0/3232705870 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00b600 con 0x7f8b5c0acd90 2023-10-06T04:33:04.501 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b43fff700 1 -- 172.21.15.123:0/3232705870 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b4c00b760 con 0x7f8b5c0acd90 2023-10-06T04:33:04.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b43fff700 1 -- 172.21.15.123:0/3232705870 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b4c00bae0 con 0x7f8b5c0acd90 2023-10-06T04:33:04.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 -- 172.21.15.123:0/3232705870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b5c097090 con 0x7f8b5c0acd90 2023-10-06T04:33:04.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.496+0000 7f8b62694700 1 -- 172.21.15.123:0/3232705870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5c0cbe50 con 0x7f8b5c0acd90 2023-10-06T04:33:04.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.497+0000 7f8b62694700 1 -- 172.21.15.123:0/3232705870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5c0b2b20 con 0x7f8b5c0acd90 2023-10-06T04:33:04.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.497+0000 7f8b43fff700 1 -- 172.21.15.123:0/3232705870 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8b4c01e460 con 0x7f8b5c0acd90 2023-10-06T04:33:04.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.498+0000 7f8b43fff700 1 --2- 172.21.15.123:0/3232705870 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8b48066f30 0x7f8b480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.502 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.498+0000 7f8b43fff700 1 -- 172.21.15.123:0/3232705870 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f8b4c084dd0 con 0x7f8b5c0acd90 2023-10-06T04:33:04.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.498+0000 7f8b5b7fe700 1 --2- 172.21.15.123:0/3232705870 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8b48066f30 0x7f8b480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.503 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.498+0000 7f8b5b7fe700 1 --2- 172.21.15.123:0/3232705870 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8b48066f30 0x7f8b480693e0 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0aba40 tx=0x7f8b50007b70 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:04.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.502+0000 7f8b43fff700 1 -- 172.21.15.123:0/3232705870 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8b4c051e80 con 0x7f8b5c0acd90 2023-10-06T04:33:04.685 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.684+0000 7f8b62694700 1 -- 172.21.15.123:0/3232705870 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8b5c04bb80 con 0x7f8b5c0acd90 2023-10-06T04:33:04.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.684+0000 7f8b43fff700 1 -- 172.21.15.123:0/3232705870 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=-16 pool 'cephfs_data' is in use by CephFS v43) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8b4c086020 con 0x7f8b5c0acd90 2023-10-06T04:33:04.687 INFO:teuthology.orchestra.run.smithi123.stderr:Error EBUSY: pool 'cephfs_data' is in use by CephFS 2023-10-06T04:33:04.691 DEBUG:teuthology.orchestra.run:got remote process result: 16 2023-10-06T04:33:04.691 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:33:04.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.807+0000 7fa5a2832700 1 Processor -- start 2023-10-06T04:33:04.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.807+0000 7fa5a2832700 1 -- start start 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa5a2832700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fa59c156f60 0x7fa59c15ad50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa5a2832700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa59c15b290 0x7fa59c0ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa5a2832700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa59c157910 con 0x7fa59c06cd60 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa5a2832700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c06d240 con 0x7fa59c15b290 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa5a2832700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa59c06d520 con 0x7fa59c156f60 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa59c15b290 0x7fa59c0ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa5a0dcf700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fa59c06cd60 0x7fa59c157da0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60520/0 (socket says 172.21.15.123:60520) 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59b7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fa59c15b290 0x7fa59c0ad730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53292/0 (socket says 172.21.15.123:53292) 2023-10-06T04:33:04.811 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa5a0dcf700 1 -- 172.21.15.123:0/2115745343 learned_addr learned my addr 172.21.15.123:0/2115745343 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59bfff700 1 --2- 172.21.15.123:0/2115745343 >> v2:172.21.15.177:3300/0 conn(0x7fa59c156f60 0x7fa59c15ad50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59b7fe700 1 -- 172.21.15.123:0/2115745343 >> v2:172.21.15.177:3300/0 conn(0x7fa59c156f60 msgr2=0x7fa59c15ad50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59b7fe700 1 --2- 172.21.15.123:0/2115745343 >> v2:172.21.15.177:3300/0 conn(0x7fa59c156f60 0x7fa59c15ad50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59b7fe700 1 -- 172.21.15.123:0/2115745343 >> v1:172.21.15.123:6789/0 conn(0x7fa59c06cd60 legacy=0x7fa59c157da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59b7fe700 1 -- 172.21.15.123:0/2115745343 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa59c06d800 con 0x7fa59c15b290 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59bfff700 1 --2- 172.21.15.123:0/2115745343 >> v2:172.21.15.177:3300/0 conn(0x7fa59c156f60 0x7fa59c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59b7fe700 1 --2- 172.21.15.123:0/2115745343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c15b290 0x7fa59c0ad730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098e0 tx=0x7fa58c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3526f6b94d84f7f1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.808+0000 7fa59affd700 1 -- 172.21.15.123:0/2115745343 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c009e80 con 0x7fa59c15b290 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa59affd700 1 -- 172.21.15.123:0/2115745343 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c00b820 con 0x7fa59c15b290 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa59affd700 1 -- 172.21.15.123:0/2115745343 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00bba0 con 0x7fa59c15b290 2023-10-06T04:33:04.812 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 -- 172.21.15.123:0/2115745343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c15b290 msgr2=0x7fa59c0ad730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 --2- 172.21.15.123:0/2115745343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c15b290 0x7fa59c0ad730 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fa58c0098e0 tx=0x7fa58c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 -- 172.21.15.123:0/2115745343 shutdown_connections 2023-10-06T04:33:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 --2- 172.21.15.123:0/2115745343 >> v2:172.21.15.177:3300/0 conn(0x7fa59c156f60 0x7fa59c15ad50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 --2- 172.21.15.123:0/2115745343 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c15b290 0x7fa59c0ad730 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 -- 172.21.15.123:0/2115745343 >> 172.21.15.123:0/2115745343 conn(0x7fa59c0ce0c0 msgr2=0x7fa59c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 -- 172.21.15.123:0/2115745343 shutdown_connections 2023-10-06T04:33:04.813 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.809+0000 7fa5a2832700 1 -- 172.21.15.123:0/2115745343 wait complete. 2023-10-06T04:33:04.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.810+0000 7fa5a2832700 1 Processor -- start 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.810+0000 7fa5a2832700 1 -- start start 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa5a2832700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa59c06cd60 0x7fa59c16a240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa5a2832700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c156f60 0x7fa59c069b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa5a2832700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c15b290 0x7fa59c16a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa5a2832700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c0696d0 con 0x7fa59c156f60 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa5a2832700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c068ac0 con 0x7fa59c15b290 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa5a2832700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59c068da0 con 0x7fa59c06cd60 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa59b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c156f60 0x7fa59c069b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa59b7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c156f60 0x7fa59c069b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53298/0 (socket says 172.21.15.123:53298) 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa59b7fe700 1 -- 172.21.15.123:0/51760520 learned_addr learned my addr 172.21.15.123:0/51760520 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa5a0dcf700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c15b290 0x7fa59c16a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa59bfff700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa59c06cd60 0x7fa59c16a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa59b7fe700 1 -- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa59c06cd60 msgr2=0x7fa59c16a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.811+0000 7fa59b7fe700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa59c06cd60 0x7fa59c16a240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa59b7fe700 1 -- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c15b290 msgr2=0x7fa59c16a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa59b7fe700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c15b290 0x7fa59c16a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa59b7fe700 1 -- 172.21.15.123:0/51760520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa58c009420 con 0x7fa59c156f60 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa59b7fe700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c156f60 0x7fa59c069b60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa58c013000 tx=0x7fa58c017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa583fff700 1 -- 172.21.15.123:0/51760520 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c00bae0 con 0x7fa59c156f60 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa583fff700 1 -- 172.21.15.123:0/51760520 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa58c025420 con 0x7fa59c156f60 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa59c069080 con 0x7fa59c156f60 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa583fff700 1 -- 172.21.15.123:0/51760520 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa58c0257e0 con 0x7fa59c156f60 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.812+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa59c16a780 con 0x7fa59c156f60 2023-10-06T04:33:04.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.813+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa59c069840 con 0x7fa59c156f60 2023-10-06T04:33:04.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.813+0000 7fa583fff700 1 -- 172.21.15.123:0/51760520 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fa58c023070 con 0x7fa59c156f60 2023-10-06T04:33:04.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.814+0000 7fa583fff700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa588066f30 0x7fa5880693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:04.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.814+0000 7fa59bfff700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa588066f30 0x7fa5880693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:04.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.814+0000 7fa583fff700 1 -- 172.21.15.123:0/51760520 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fa58c085d60 con 0x7fa59c156f60 2023-10-06T04:33:04.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.814+0000 7fa59bfff700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa588066f30 0x7fa5880693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b2ba0 tx=0x7fa590006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:04.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:04.818+0000 7fa583fff700 1 -- 172.21.15.123:0/51760520 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fa58c052e10 con 0x7fa59c156f60 2023-10-06T04:33:05.004 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.003+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa59c06dae0 con 0x7fa59c156f60 2023-10-06T04:33:05.005 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.004+0000 7fa583fff700 1 -- 172.21.15.123:0/51760520 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17156 (secure 0 0 0) 0x7fa58c015090 con 0x7fa59c156f60 2023-10-06T04:33:05.005 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:05.005 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:33:04.304393+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:33:05.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa588066f30 msgr2=0x7fa5880693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa588066f30 0x7fa5880693e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fa59c0b2ba0 tx=0x7fa590006f90 comp rx=0 tx=0).stop 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c156f60 msgr2=0x7fa59c069b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c156f60 0x7fa59c069b60 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fa58c013000 tx=0x7fa58c017000 comp rx=0 tx=0).stop 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 shutdown_connections 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fa588066f30 0x7fa5880693e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fa59c15b290 0x7fa59c16a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fa59c06cd60 0x7fa59c16a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 --2- 172.21.15.123:0/51760520 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fa59c156f60 0x7fa59c069b60 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 >> 172.21.15.123:0/51760520 conn(0x7fa59c0ce0c0 msgr2=0x7fa59c1553d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 shutdown_connections 2023-10-06T04:33:05.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.006+0000 7fa5a2832700 1 -- 172.21.15.123:0/51760520 wait complete. 2023-10-06T04:33:05.019 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-10-06T04:31:20.921532+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '16', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}}, {'pool': 2, 'pool_name': 'rbd', 'create_time': '2023-10-06T04:31:27.955056+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 8, 'pg_placement_num': 8, 'pg_placement_num_target': 8, 'pg_num_target': 8, 'pg_num_pending': 8, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '19', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {}, 'application_metadata': {'rbd': {}}}, {'pool': 6, 'pool_name': 'cephfs_data', 'create_time': '2023-10-06T04:32:22.782310+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '41', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'recovery_fs'}}}, {'pool': 7, 'pool_name': 'recovery_fs_metadata', 'create_time': '2023-10-06T04:32:35.940980+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '42', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'recovery_fs'}}}] 2023-10-06T04:33:05.020 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-10-06T04:33:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.140+0000 7f660bcd2700 1 Processor -- start 2023-10-06T04:33:05.142 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.141+0000 7f660bcd2700 1 -- start start 2023-10-06T04:33:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.141+0000 7f660bcd2700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f660413fa70 0x7f66040c8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.141+0000 7f660bcd2700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f660406e9f0 0x7f66040c8840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.141+0000 7f660bcd2700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f660406a4e0 con 0x7f66040cabf0 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.141+0000 7f660bcd2700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040c8ec0 con 0x7f660413fa70 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f660bcd2700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66040c91a0 con 0x7f660406e9f0 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f660413fa70 0x7f66040c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f660413fa70 0x7f66040c8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53304/0 (socket says 172.21.15.123:53304) 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 -- 172.21.15.123:0/129897591 learned_addr learned my addr 172.21.15.123:0/129897591 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 -- 172.21.15.123:0/129897591 >> v2:172.21.15.177:3300/0 conn(0x7f660406e9f0 msgr2=0x7f66040c8840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 --2- 172.21.15.123:0/129897591 >> v2:172.21.15.177:3300/0 conn(0x7f660406e9f0 0x7f66040c8840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 -- 172.21.15.123:0/129897591 >> v1:172.21.15.177:6789/0 conn(0x7f66040cabf0 legacy=0x7f660406e2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 -- 172.21.15.123:0/129897591 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66040c9480 con 0x7f660413fa70 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6609a6e700 1 --2- 172.21.15.123:0/129897591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660413fa70 0x7f66040c8300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f66040cdf90 tx=0x7f65f8009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc0905d6389be5b2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.145 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6608a6c700 1 -- 172.21.15.123:0/129897591 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800b820 con 0x7f660413fa70 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6608a6c700 1 -- 172.21.15.123:0/129897591 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f800b980 con 0x7f660413fa70 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.142+0000 7f6608a6c700 1 -- 172.21.15.123:0/129897591 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f800bd20 con 0x7f660413fa70 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 -- 172.21.15.123:0/129897591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660413fa70 msgr2=0x7f66040c8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 --2- 172.21.15.123:0/129897591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660413fa70 0x7f66040c8300 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f66040cdf90 tx=0x7f65f8009250 comp rx=0 tx=0).stop 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 -- 172.21.15.123:0/129897591 shutdown_connections 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 --2- 172.21.15.123:0/129897591 >> v2:172.21.15.177:3300/0 conn(0x7f660406e9f0 0x7f66040c8840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 --2- 172.21.15.123:0/129897591 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660413fa70 0x7f66040c8300 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 -- 172.21.15.123:0/129897591 >> 172.21.15.123:0/129897591 conn(0x7f66040cd020 msgr2=0x7f66040cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 -- 172.21.15.123:0/129897591 shutdown_connections 2023-10-06T04:33:05.146 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.143+0000 7f660bcd2700 1 -- 172.21.15.123:0/129897591 wait complete. 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.144+0000 7f660bcd2700 1 Processor -- start 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.144+0000 7f660bcd2700 1 -- start start 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f660bcd2700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660406e9f0 0x7f66040c6ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f660bcd2700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66040cabf0 0x7f66040ac710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f660bcd2700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f660413fa70 0x7f66040ad0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f660bcd2700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c4dd0 con 0x7f660406e9f0 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f660bcd2700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66040c4200 con 0x7f66040cabf0 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f660bcd2700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66040acc50 con 0x7f660413fa70 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f6609a6e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660406e9f0 0x7f66040c6ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f6609a6e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660406e9f0 0x7f66040c6ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53310/0 (socket says 172.21.15.123:53310) 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.145+0000 7f6609a6e700 1 -- 172.21.15.123:0/85318815 learned_addr learned my addr 172.21.15.123:0/85318815 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f660926d700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66040cabf0 0x7f66040ac710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f660a26f700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f660413fa70 0x7f66040ad0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f6609a6e700 1 -- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f660413fa70 msgr2=0x7f66040ad0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f6609a6e700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f660413fa70 0x7f66040ad0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f6609a6e700 1 -- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66040cabf0 msgr2=0x7f66040ac710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f6609a6e700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66040cabf0 0x7f66040ac710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f6609a6e700 1 -- 172.21.15.123:0/85318815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65f80094f0 con 0x7f660406e9f0 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f660a26f700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f660413fa70 0x7f66040ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f660926d700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66040cabf0 0x7f66040ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:05.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f6609a6e700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660406e9f0 0x7f66040c6ee0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f65f8003080 tx=0x7f65f8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f65f5ffb700 1 -- 172.21.15.123:0/85318815 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f801a460 con 0x7f660406e9f0 2023-10-06T04:33:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f65f5ffb700 1 -- 172.21.15.123:0/85318815 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65f801a5e0 con 0x7f660406e9f0 2023-10-06T04:33:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f65f5ffb700 1 -- 172.21.15.123:0/85318815 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65f801a9a0 con 0x7f660406e9f0 2023-10-06T04:33:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66040c44a0 con 0x7f660406e9f0 2023-10-06T04:33:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.146+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66040c4a60 con 0x7f660406e9f0 2023-10-06T04:33:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.147+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f660406d240 con 0x7f660406e9f0 2023-10-06T04:33:05.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.148+0000 7f65f5ffb700 1 -- 172.21.15.123:0/85318815 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f65f801ad50 con 0x7f660406e9f0 2023-10-06T04:33:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.148+0000 7f65f5ffb700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f65e8066f30 0x7f65e80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.148+0000 7f65f5ffb700 1 -- 172.21.15.123:0/85318815 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f65f8086070 con 0x7f660406e9f0 2023-10-06T04:33:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.148+0000 7f660926d700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f65e8066f30 0x7f65e80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.152 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.148+0000 7f660926d700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f65e8066f30 0x7f65e80693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6600006fd0 tx=0x7f6600008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.152+0000 7f65f5ffb700 1 -- 172.21.15.123:0/85318815 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f65f8053120 con 0x7f660406e9f0 2023-10-06T04:33:05.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.336+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f66040c3d10 con 0x7f660406e9f0 2023-10-06T04:33:05.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.336+0000 7f65f5ffb700 1 -- 172.21.15.123:0/85318815 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v43) v1 ==== 93+0+10 (secure 0 0 0) 0x7f65f80200a0 con 0x7f660406e9f0 2023-10-06T04:33:05.338 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 1 2023-10-06T04:33:05.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f65e8066f30 msgr2=0x7f65e80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f65e8066f30 0x7f65e80693e0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f6600006fd0 tx=0x7f6600008040 comp rx=0 tx=0).stop 2023-10-06T04:33:05.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660406e9f0 msgr2=0x7f66040c6ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660406e9f0 0x7f66040c6ee0 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f65f8003080 tx=0x7f65f8016000 comp rx=0 tx=0).stop 2023-10-06T04:33:05.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 shutdown_connections 2023-10-06T04:33:05.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f65e8066f30 0x7f65e80693e0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.340 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f660406e9f0 0x7f66040c6ee0 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f660413fa70 0x7f66040ad0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 --2- 172.21.15.123:0/85318815 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f66040cabf0 0x7f66040ac710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 >> 172.21.15.123:0/85318815 conn(0x7f66040cd020 msgr2=0x7f66040c24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:05.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 shutdown_connections 2023-10-06T04:33:05.341 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.338+0000 7f660bcd2700 1 -- 172.21.15.123:0/85318815 wait complete. 2023-10-06T04:33:05.351 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2023-10-06T04:33:05.471 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.470+0000 7f105c3dc700 1 Processor -- start 2023-10-06T04:33:05.472 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.471+0000 7f105c3dc700 1 -- start start 2023-10-06T04:33:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f105c3dc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f105413fdc0 0x7f1054067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f105c3dc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1054067ba0 0x7f10540c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.474 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f105c3dc700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105406b020 con 0x7f105413f9f0 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f105c3dc700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1054068020 con 0x7f105413fdc0 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f105c3dc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105406a450 con 0x7f1054067ba0 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f1059977700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f105413fdc0 0x7f1054067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f1059977700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f105413fdc0 0x7f1054067660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53318/0 (socket says 172.21.15.123:53318) 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f1059977700 1 -- 172.21.15.123:0/1383713204 learned_addr learned my addr 172.21.15.123:0/1383713204 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f105a979700 1 --2- 172.21.15.123:0/1383713204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1054067ba0 0x7f10540c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f1059977700 1 -- 172.21.15.123:0/1383713204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1054067ba0 msgr2=0x7f10540c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f1059977700 1 --2- 172.21.15.123:0/1383713204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1054067ba0 0x7f10540c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f1059977700 1 -- 172.21.15.123:0/1383713204 >> v1:172.21.15.177:6789/0 conn(0x7f105413f9f0 legacy=0x7f105406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f1059977700 1 -- 172.21.15.123:0/1383713204 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105406bbc0 con 0x7f105413fdc0 2023-10-06T04:33:05.475 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.472+0000 7f105a979700 1 --2- 172.21.15.123:0/1383713204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1054067ba0 0x7f10540c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:05.476 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.473+0000 7f1059977700 1 --2- 172.21.15.123:0/1383713204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f105413fdc0 0x7f1054067660 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f10440099b0 tx=0x7f1044009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78060b97c9afe54c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.476 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.473+0000 7f1059176700 1 -- 172.21.15.123:0/1383713204 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400b860 con 0x7f105413fdc0 2023-10-06T04:33:05.476 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.473+0000 7f1059176700 1 -- 172.21.15.123:0/1383713204 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104400b9c0 con 0x7f105413fdc0 2023-10-06T04:33:05.476 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.473+0000 7f1059176700 1 -- 172.21.15.123:0/1383713204 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104400bd60 con 0x7f105413fdc0 2023-10-06T04:33:05.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 -- 172.21.15.123:0/1383713204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f105413fdc0 msgr2=0x7f1054067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 --2- 172.21.15.123:0/1383713204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f105413fdc0 0x7f1054067660 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f10440099b0 tx=0x7f1044009250 comp rx=0 tx=0).stop 2023-10-06T04:33:05.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 -- 172.21.15.123:0/1383713204 shutdown_connections 2023-10-06T04:33:05.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 --2- 172.21.15.123:0/1383713204 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1054067ba0 0x7f10540c7660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 --2- 172.21.15.123:0/1383713204 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f105413fdc0 0x7f1054067660 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.478 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 -- 172.21.15.123:0/1383713204 >> 172.21.15.123:0/1383713204 conn(0x7f10540cd020 msgr2=0x7f10540cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:05.478 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 -- 172.21.15.123:0/1383713204 shutdown_connections 2023-10-06T04:33:05.478 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.474+0000 7f105c3dc700 1 -- 172.21.15.123:0/1383713204 wait complete. 2023-10-06T04:33:05.478 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.475+0000 7f105c3dc700 1 Processor -- start 2023-10-06T04:33:05.478 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.475+0000 7f105c3dc700 1 -- start start 2023-10-06T04:33:05.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105c3dc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1054067ba0 0x7f10541566d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105c3dc700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f105413f9f0 0x7f105413c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105c3dc700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105413db20 0x7f10540afde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105c3dc700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105413f030 con 0x7f1054067ba0 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105c3dc700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105413eba0 con 0x7f105413db20 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105c3dc700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f105413c5c0 con 0x7f105413f9f0 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105a178700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1054067ba0 0x7f10541566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105a178700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1054067ba0 0x7f10541566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53332/0 (socket says 172.21.15.123:53332) 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.476+0000 7f105a178700 1 -- 172.21.15.123:0/47852808 learned_addr learned my addr 172.21.15.123:0/47852808 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f1059977700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f105413f9f0 0x7f105413c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105a979700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105413db20 0x7f10540afde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105a178700 1 -- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f105413f9f0 msgr2=0x7f105413c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105a178700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f105413f9f0 0x7f105413c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105a178700 1 -- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105413db20 msgr2=0x7f10540afde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105a178700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105413db20 0x7f10540afde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105a178700 1 -- 172.21.15.123:0/47852808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10440094f0 con 0x7f1054067ba0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105a178700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1054067ba0 0x7f10541566d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f104c00cb40 tx=0x7f104c0017b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f104a7fc700 1 -- 172.21.15.123:0/47852808 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c00a7a0 con 0x7f1054067ba0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f104a7fc700 1 -- 172.21.15.123:0/47852808 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c00a920 con 0x7f1054067ba0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f104a7fc700 1 -- 172.21.15.123:0/47852808 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c00ace0 con 0x7f1054067ba0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f105413df90 con 0x7f1054067ba0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.477+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f105413e550 con 0x7f1054067ba0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.478+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105406d240 con 0x7f1054067ba0 2023-10-06T04:33:05.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.478+0000 7f104a7fc700 1 -- 172.21.15.123:0/47852808 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f104c003030 con 0x7f1054067ba0 2023-10-06T04:33:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.479+0000 7f104a7fc700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.479+0000 7f104a7fc700 1 -- 172.21.15.123:0/47852808 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f104c0847d0 con 0x7f1054067ba0 2023-10-06T04:33:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.479+0000 7f1059977700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.479+0000 7f1059977700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1038066f30 0x7f10380693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1054143490 tx=0x7f1044017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.483+0000 7f104a7fc700 1 -- 172.21.15.123:0/47852808 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f104c051880 con 0x7f1054067ba0 2023-10-06T04:33:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.664+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"} v 0) v1 -- 0x7f105413ee80 con 0x7f1054067ba0 2023-10-06T04:33:05.666 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.665+0000 7f104a7fc700 1 -- 172.21.15.123:0/47852808 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "rbd", "var": "pg_num"}]=0 v43) v1 ==== 92+0+10 (secure 0 0 0) 0x7f104c054ea0 con 0x7f1054067ba0 2023-10-06T04:33:05.666 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 8 2023-10-06T04:33:05.668 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1038066f30 msgr2=0x7f10380693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1038066f30 0x7f10380693e0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f1054143490 tx=0x7f1044017000 comp rx=0 tx=0).stop 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1054067ba0 msgr2=0x7f10541566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1054067ba0 0x7f10541566d0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7f104c00cb40 tx=0x7f104c0017b0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 shutdown_connections 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1038066f30 0x7f10380693e0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1054067ba0 0x7f10541566d0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f105413db20 0x7f10540afde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 --2- 172.21.15.123:0/47852808 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f105413f9f0 0x7f105413c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.669 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 >> 172.21.15.123:0/47852808 conn(0x7f10540cd020 msgr2=0x7f10540b2860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:05.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 shutdown_connections 2023-10-06T04:33:05.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.667+0000 7f105c3dc700 1 -- 172.21.15.123:0/47852808 wait complete. 2023-10-06T04:33:05.680 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-10-06T04:33:05.799 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.798+0000 7feb4beee700 1 Processor -- start 2023-10-06T04:33:05.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.798+0000 7feb4beee700 1 -- start start 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb4beee700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb440af600 0x7feb440b1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb4beee700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7feb440ae9c0 0x7feb44069d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb4beee700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb44069880 con 0x7feb44068770 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb4beee700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb440cdfd0 con 0x7feb440af600 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb4beee700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb44068c70 con 0x7feb440ae9c0 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49c8a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb440af600 0x7feb440b1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49c8a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7feb440af600 0x7feb440b1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53346/0 (socket says 172.21.15.123:53346) 2023-10-06T04:33:05.802 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49c8a700 1 -- 172.21.15.123:0/1616533492 learned_addr learned my addr 172.21.15.123:0/1616533492 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49489700 1 --2- 172.21.15.123:0/1616533492 >> v2:172.21.15.177:3300/0 conn(0x7feb440ae9c0 0x7feb44069d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49c8a700 1 -- 172.21.15.123:0/1616533492 >> v2:172.21.15.177:3300/0 conn(0x7feb440ae9c0 msgr2=0x7feb44069d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49c8a700 1 --2- 172.21.15.123:0/1616533492 >> v2:172.21.15.177:3300/0 conn(0x7feb440ae9c0 0x7feb44069d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49c8a700 1 -- 172.21.15.123:0/1616533492 >> v1:172.21.15.177:6789/0 conn(0x7feb44068770 legacy=0x7feb44158fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.799+0000 7feb49c8a700 1 -- 172.21.15.123:0/1616533492 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb44068f50 con 0x7feb440af600 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb49c8a700 1 --2- 172.21.15.123:0/1616533492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb440af600 0x7feb440b1830 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feb34009990 tx=0x7feb34009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=18c5ffa37b0cead3 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb48c88700 1 -- 172.21.15.123:0/1616533492 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34005c10 con 0x7feb440af600 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb48c88700 1 -- 172.21.15.123:0/1616533492 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34005d70 con 0x7feb440af600 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb48c88700 1 -- 172.21.15.123:0/1616533492 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb3400b800 con 0x7feb440af600 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 -- 172.21.15.123:0/1616533492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb440af600 msgr2=0x7feb440b1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.803 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 --2- 172.21.15.123:0/1616533492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb440af600 0x7feb440b1830 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7feb34009990 tx=0x7feb34009220 comp rx=0 tx=0).stop 2023-10-06T04:33:05.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 -- 172.21.15.123:0/1616533492 shutdown_connections 2023-10-06T04:33:05.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 --2- 172.21.15.123:0/1616533492 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb440af600 0x7feb440b1830 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 --2- 172.21.15.123:0/1616533492 >> v2:172.21.15.177:3300/0 conn(0x7feb440ae9c0 0x7feb44069d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 -- 172.21.15.123:0/1616533492 >> 172.21.15.123:0/1616533492 conn(0x7feb440973b0 msgr2=0x7feb44067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:05.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 -- 172.21.15.123:0/1616533492 shutdown_connections 2023-10-06T04:33:05.804 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.800+0000 7feb4beee700 1 -- 172.21.15.123:0/1616533492 wait complete. 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.801+0000 7feb4beee700 1 Processor -- start 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4beee700 1 -- start start 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4beee700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb44068770 0x7feb441597b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4beee700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb440ae9c0 0x7feb4409b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4beee700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb440af600 0x7feb44169730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4beee700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4409b110 con 0x7feb44068770 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4beee700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb4409a500 con 0x7feb440af600 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4beee700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb4409a7e0 con 0x7feb440ae9c0 2023-10-06T04:33:05.807 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb49c8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb44068770 0x7feb441597b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb49c8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb44068770 0x7feb441597b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53354/0 (socket says 172.21.15.123:53354) 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb49c8a700 1 -- 172.21.15.123:0/3987119820 learned_addr learned my addr 172.21.15.123:0/3987119820 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb49489700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb440ae9c0 0x7feb4409b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.802+0000 7feb4a48b700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb440af600 0x7feb44169730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb49c8a700 1 -- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb440ae9c0 msgr2=0x7feb4409b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb49c8a700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb440ae9c0 0x7feb4409b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb49c8a700 1 -- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb440af600 msgr2=0x7feb44169730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb49c8a700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb440af600 0x7feb44169730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb49c8a700 1 -- 172.21.15.123:0/3987119820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb34009480 con 0x7feb44068770 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb49c8a700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb44068770 0x7feb441597b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb340030c0 tx=0x7feb34015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.808 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb39ffb700 1 -- 172.21.15.123:0/3987119820 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb340118b0 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb39ffb700 1 -- 172.21.15.123:0/3987119820 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb34011a10 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb4409aac0 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb39ffb700 1 -- 172.21.15.123:0/3987119820 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb34011dd0 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.803+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb4409bae0 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.804+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb44069230 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.805+0000 7feb39ffb700 1 -- 172.21.15.123:0/3987119820 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7feb3401c630 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.805+0000 7feb39ffb700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feb28066f30 0x7feb280693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.805+0000 7feb39ffb700 1 -- 172.21.15.123:0/3987119820 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7feb34086150 con 0x7feb44068770 2023-10-06T04:33:05.809 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.805+0000 7feb49489700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feb28066f30 0x7feb280693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:05.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.805+0000 7feb49489700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feb28066f30 0x7feb280693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feb440c3460 tx=0x7feb40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:05.810 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.809+0000 7feb39ffb700 1 -- 172.21.15.123:0/3987119820 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7feb34053200 con 0x7feb44068770 2023-10-06T04:33:05.994 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.993+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7feb4404bb80 con 0x7feb44068770 2023-10-06T04:33:05.995 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.993+0000 7feb39ffb700 1 -- 172.21.15.123:0/3987119820 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v43) v1 ==== 100+0+11 (secure 0 0 0) 0x7feb34056820 con 0x7feb44068770 2023-10-06T04:33:05.995 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 64 2023-10-06T04:33:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feb28066f30 msgr2=0x7feb280693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feb28066f30 0x7feb280693e0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7feb440c3460 tx=0x7feb40008040 comp rx=0 tx=0).stop 2023-10-06T04:33:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb44068770 msgr2=0x7feb441597b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb44068770 0x7feb441597b0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7feb340030c0 tx=0x7feb34015000 comp rx=0 tx=0).stop 2023-10-06T04:33:05.997 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 shutdown_connections 2023-10-06T04:33:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7feb28066f30 0x7feb280693e0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7feb44068770 0x7feb441597b0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7feb440af600 0x7feb44169730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 --2- 172.21.15.123:0/3987119820 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7feb440ae9c0 0x7feb4409b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 >> 172.21.15.123:0/3987119820 conn(0x7feb440973b0 msgr2=0x7feb4415b090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 shutdown_connections 2023-10-06T04:33:05.998 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:05.995+0000 7feb4beee700 1 -- 172.21.15.123:0/3987119820 wait complete. 2023-10-06T04:33:06.008 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get recovery_fs_metadata pg_num 2023-10-06T04:33:06.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.127+0000 7fe8af54e700 1 Processor -- start 2023-10-06T04:33:06.128 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.127+0000 7fe8af54e700 1 -- start start 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8af54e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8a8157430 0x7fe8a80ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8af54e700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a80c9d70 con 0x7fe8a80caf00 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8af54e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8a8159200 con 0x7fe8a80c9150 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8af54e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8a8157800 con 0x7fe8a8157430 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8ad2ea700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe8a8157430 0x7fe8a80ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8acae9700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fe8a80caf00 0x7fe8a80c8a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60526/0 (socket says 172.21.15.123:60526) 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8acae9700 1 -- 172.21.15.123:0/1012112423 learned_addr learned my addr 172.21.15.123:0/1012112423 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8ad2ea700 1 -- 172.21.15.123:0/1012112423 >> v1:172.21.15.177:6789/0 conn(0x7fe8a80c9150 legacy=0x7fe8a8159690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8ad2ea700 1 -- 172.21.15.123:0/1012112423 >> v1:172.21.15.123:6789/0 conn(0x7fe8a80caf00 legacy=0x7fe8a80c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8ad2ea700 1 -- 172.21.15.123:0/1012112423 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a80c9550 con 0x7fe8a8157430 2023-10-06T04:33:06.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe89ffff700 1 -- 172.21.15.123:0/1012112423 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1278414761 0 0) 0x7fe898001110 con 0x7fe8a80caf00 2023-10-06T04:33:06.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe8ad2ea700 1 --2- 172.21.15.123:0/1012112423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80ca9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40098d0 tx=0x7fe8a4009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3eaf5e3fdbecc219 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe89ffff700 1 -- 172.21.15.123:0/1012112423 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a4009e70 con 0x7fe8a8157430 2023-10-06T04:33:06.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.128+0000 7fe89ffff700 1 -- 172.21.15.123:0/1012112423 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a4005c10 con 0x7fe8a8157430 2023-10-06T04:33:06.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe89ffff700 1 -- 172.21.15.123:0/1012112423 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a400b610 con 0x7fe8a8157430 2023-10-06T04:33:06.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe8af54e700 1 -- 172.21.15.123:0/1012112423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 msgr2=0x7fe8a80ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.133 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1012112423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80ca9c0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7fe8a40098d0 tx=0x7fe8a4009120 comp rx=0 tx=0).stop 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe8af54e700 1 -- 172.21.15.123:0/1012112423 shutdown_connections 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1012112423 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80ca9c0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe8af54e700 1 -- 172.21.15.123:0/1012112423 >> 172.21.15.123:0/1012112423 conn(0x7fe8a8097450 msgr2=0x7fe8a8067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe8af54e700 1 -- 172.21.15.123:0/1012112423 shutdown_connections 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.129+0000 7fe8af54e700 1 -- 172.21.15.123:0/1012112423 wait complete. 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.130+0000 7fe8af54e700 1 Processor -- start 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.130+0000 7fe8af54e700 1 -- start start 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8af54e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80c9150 0x7fe8a816d190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8af54e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a80caf00 0x7fe8a80af8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8af54e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80cce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.134 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8af54e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a80c4130 con 0x7fe8a8157430 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8af54e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a816d6d0 con 0x7fe8a80caf00 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8af54e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8a816d970 con 0x7fe8a80c9150 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8adaeb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8adaeb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80cce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53380/0 (socket says 172.21.15.123:53380) 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8adaeb700 1 -- 172.21.15.123:0/1504278252 learned_addr learned my addr 172.21.15.123:0/1504278252 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8ad2ea700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80c9150 0x7fe8a816d190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.131+0000 7fe8acae9700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a80caf00 0x7fe8a80af8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8adaeb700 1 -- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80c9150 msgr2=0x7fe8a816d190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8adaeb700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80c9150 0x7fe8a816d190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8adaeb700 1 -- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a80caf00 msgr2=0x7fe8a80af8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8adaeb700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a80caf00 0x7fe8a80af8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.135 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8adaeb700 1 -- 172.21.15.123:0/1504278252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8a40093c0 con 0x7fe8a8157430 2023-10-06T04:33:06.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8adaeb700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80cce80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c930 tx=0x7fe8a00026e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe89d7fa700 1 -- 172.21.15.123:0/1504278252 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0012630 con 0x7fe8a8157430 2023-10-06T04:33:06.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe89d7fa700 1 -- 172.21.15.123:0/1504278252 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8a00127d0 con 0x7fe8a8157430 2023-10-06T04:33:06.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8a816dc50 con 0x7fe8a8157430 2023-10-06T04:33:06.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe89d7fa700 1 -- 172.21.15.123:0/1504278252 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8a0012b90 con 0x7fe8a8157430 2023-10-06T04:33:06.136 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.132+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8a80c3d50 con 0x7fe8a8157430 2023-10-06T04:33:06.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.133+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe8a80c42a0 con 0x7fe8a8157430 2023-10-06T04:33:06.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.133+0000 7fe89d7fa700 1 -- 172.21.15.123:0/1504278252 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe8a001a3b0 con 0x7fe8a8157430 2023-10-06T04:33:06.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.134+0000 7fe89d7fa700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe88c066f30 0x7fe88c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.134+0000 7fe89d7fa700 1 -- 172.21.15.123:0/1504278252 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fe8a0084340 con 0x7fe8a8157430 2023-10-06T04:33:06.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.134+0000 7fe8ad2ea700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe88c066f30 0x7fe88c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.137 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.134+0000 7fe8ad2ea700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe88c066f30 0x7fe88c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4003080 tx=0x7fe8a4015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.139 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.138+0000 7fe89d7fa700 1 -- 172.21.15.123:0/1504278252 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe8a0054e80 con 0x7fe8a8157430 2023-10-06T04:33:06.319 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.318+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe8a806d840 con 0x7fe8a8157430 2023-10-06T04:33:06.322 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.321+0000 7fe89d7fa700 1 -- 172.21.15.123:0/1504278252 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "recovery_fs_metadata", "var": "pg_num"}]=0 v43) v1 ==== 109+0+11 (secure 0 0 0) 0x7fe8a0054a10 con 0x7fe8a8157430 2023-10-06T04:33:06.323 INFO:teuthology.orchestra.run.smithi123.stdout:pg_num: 32 2023-10-06T04:33:06.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe88c066f30 msgr2=0x7fe88c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe88c066f30 0x7fe88c0693e0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7fe8a4003080 tx=0x7fe8a4015000 comp rx=0 tx=0).stop 2023-10-06T04:33:06.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 msgr2=0x7fe8a80cce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80cce80 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7fe8a000c930 tx=0x7fe8a00026e0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 shutdown_connections 2023-10-06T04:33:06.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe88c066f30 0x7fe88c0693e0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe8a8157430 0x7fe8a80cce80 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe8a80c9150 0x7fe8a816d190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 --2- 172.21.15.123:0/1504278252 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe8a80caf00 0x7fe8a80af8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 >> 172.21.15.123:0/1504278252 conn(0x7fe8a8097450 msgr2=0x7fe8a806b580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 shutdown_connections 2023-10-06T04:33:06.326 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.323+0000 7fe8af54e700 1 -- 172.21.15.123:0/1504278252 wait complete. 2023-10-06T04:33:06.336 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:33:06.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.451+0000 7ff97467c700 1 Processor -- start 2023-10-06T04:33:06.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.451+0000 7ff97467c700 1 -- start start 2023-10-06T04:33:06.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.452+0000 7ff97467c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff96c13f600 0x7ff96c06b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.452+0000 7ff97467c700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c06be00 con 0x7ff96c15d720 2023-10-06T04:33:06.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.452+0000 7ff97467c700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff96c0690a0 con 0x7ff96c06b990 2023-10-06T04:33:06.455 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.452+0000 7ff97467c700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff96c15db60 con 0x7ff96c13f600 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.452+0000 7ff972418700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff96c13f600 0x7ff96c06b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.452+0000 7ff972c19700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff96c15d720 0x7ff96c069530 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60534/0 (socket says 172.21.15.123:60534) 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.452+0000 7ff972c19700 1 -- 172.21.15.123:0/582058843 learned_addr learned my addr 172.21.15.123:0/582058843 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff972418700 1 -- 172.21.15.123:0/582058843 >> v1:172.21.15.177:6789/0 conn(0x7ff96c06b990 legacy=0x7ff96c15d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff972418700 1 -- 172.21.15.123:0/582058843 >> v1:172.21.15.123:6789/0 conn(0x7ff96c15d720 legacy=0x7ff96c069530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff972418700 1 -- 172.21.15.123:0/582058843 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff96c15dd40 con 0x7ff96c13f600 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff971416700 1 -- 172.21.15.123:0/582058843 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2006121905 0 0) 0x7ff96c06be00 con 0x7ff96c15d720 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff972418700 1 --2- 172.21.15.123:0/582058843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c13f600 0x7ff96c06b450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff9680098e0 tx=0x7ff9680091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=373fe6522b558752 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff971416700 1 -- 172.21.15.123:0/582058843 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff968009e80 con 0x7ff96c13f600 2023-10-06T04:33:06.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff971416700 1 -- 172.21.15.123:0/582058843 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96800b860 con 0x7ff96c13f600 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff971416700 1 -- 172.21.15.123:0/582058843 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800bbe0 con 0x7ff96c13f600 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff97467c700 1 -- 172.21.15.123:0/582058843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c13f600 msgr2=0x7ff96c06b450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.453+0000 7ff97467c700 1 --2- 172.21.15.123:0/582058843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c13f600 0x7ff96c06b450 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7ff9680098e0 tx=0x7ff9680091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.454+0000 7ff97467c700 1 -- 172.21.15.123:0/582058843 shutdown_connections 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.454+0000 7ff97467c700 1 --2- 172.21.15.123:0/582058843 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c13f600 0x7ff96c06b450 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.454+0000 7ff97467c700 1 -- 172.21.15.123:0/582058843 >> 172.21.15.123:0/582058843 conn(0x7ff96c0ce0c0 msgr2=0x7ff96c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.454+0000 7ff97467c700 1 -- 172.21.15.123:0/582058843 shutdown_connections 2023-10-06T04:33:06.457 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.454+0000 7ff97467c700 1 -- 172.21.15.123:0/582058843 wait complete. 2023-10-06T04:33:06.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 Processor -- start 2023-10-06T04:33:06.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 -- start start 2023-10-06T04:33:06.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c06b990 0x7ff96c16b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c13f600 0x7ff96c16c8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15d720 0x7ff96c16ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0c7d20 con 0x7ff96c06b990 2023-10-06T04:33:06.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0c5ba0 con 0x7ff96c13f600 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.455+0000 7ff97467c700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff96c0c5270 con 0x7ff96c15d720 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c06b990 0x7ff96c16b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c06b990 0x7ff96c16b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53402/0 (socket says 172.21.15.123:53402) 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 -- 172.21.15.123:0/1768359691 learned_addr learned my addr 172.21.15.123:0/1768359691 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972c19700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15d720 0x7ff96c16ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff971c17700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c13f600 0x7ff96c16c8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 -- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15d720 msgr2=0x7ff96c16ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15d720 0x7ff96c16ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 -- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c13f600 msgr2=0x7ff96c16c8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c13f600 0x7ff96c16c8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 -- 172.21.15.123:0/1768359691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff968009420 con 0x7ff96c06b990 2023-10-06T04:33:06.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff971c17700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c13f600 0x7ff96c16c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff972418700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c06b990 0x7ff96c16b000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff9680091e0 tx=0x7ff968016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff9627fc700 1 -- 172.21.15.123:0/1768359691 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96800ba50 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff9627fc700 1 -- 172.21.15.123:0/1768359691 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff968020070 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff96c0c5550 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff9627fc700 1 -- 172.21.15.123:0/1768359691 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96801b6a0 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.456+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff96c16b540 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.457+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff96c06e690 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.457+0000 7ff9627fc700 1 -- 172.21.15.123:0/1768359691 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff968022030 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.458+0000 7ff9627fc700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff950066f30 0x7ff9500693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.458+0000 7ff9627fc700 1 -- 172.21.15.123:0/1768359691 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7ff9680866c0 con 0x7ff96c06b990 2023-10-06T04:33:06.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.458+0000 7ff971c17700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff950066f30 0x7ff9500693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.458+0000 7ff971c17700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff950066f30 0x7ff9500693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006fd0 tx=0x7ff95c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.462+0000 7ff9627fc700 1 -- 172.21.15.123:0/1768359691 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7ff968053770 con 0x7ff96c06b990 2023-10-06T04:33:06.686 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.684+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff96c068770 con 0x7ff96c06b990 2023-10-06T04:33:06.687 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.684+0000 7ff9627fc700 1 -- 172.21.15.123:0/1768359691 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4609 (secure 0 0 0) 0x7ff968056d90 con 0x7ff96c06b990 2023-10-06T04:33:06.687 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:06.687 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:33:06.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff950066f30 msgr2=0x7ff9500693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff950066f30 0x7ff9500693e0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7ff95c006fd0 tx=0x7ff95c008040 comp rx=0 tx=0).stop 2023-10-06T04:33:06.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c06b990 msgr2=0x7ff96c16b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c06b990 0x7ff96c16b000 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7ff9680091e0 tx=0x7ff968016000 comp rx=0 tx=0).stop 2023-10-06T04:33:06.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 shutdown_connections 2023-10-06T04:33:06.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff950066f30 0x7ff9500693e0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff96c06b990 0x7ff96c16b000 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff96c15d720 0x7ff96c16ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 --2- 172.21.15.123:0/1768359691 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff96c13f600 0x7ff96c16c8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 >> 172.21.15.123:0/1768359691 conn(0x7ff96c0ce0c0 msgr2=0x7ff96c0cc490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:06.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 shutdown_connections 2023-10-06T04:33:06.689 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.686+0000 7ff97467c700 1 -- 172.21.15.123:0/1768359691 wait complete. 2023-10-06T04:33:06.690 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2023-10-06T04:33:06.699 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:33:06.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.816+0000 7f0186e3e700 1 Processor -- start 2023-10-06T04:33:06.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.816+0000 7f0186e3e700 1 -- start start 2023-10-06T04:33:06.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.816+0000 7f0186e3e700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f01800aae60 0x7f01800aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.816+0000 7f0186e3e700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f01800ace40 0x7f01800ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.816+0000 7f0186e3e700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01800abe30 con 0x7f01800ab230 2023-10-06T04:33:06.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.816+0000 7f0186e3e700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01800ac9b0 con 0x7f01800aae60 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.816+0000 7f0186e3e700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01800ac080 con 0x7f01800ace40 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f01800aae60 0x7f01800aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f01800aae60 0x7f01800aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53404/0 (socket says 172.21.15.123:53404) 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 -- 172.21.15.123:0/2487841464 learned_addr learned my addr 172.21.15.123:0/2487841464 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f01853db700 1 --2- 172.21.15.123:0/2487841464 >> v2:172.21.15.177:3300/0 conn(0x7f01800ace40 0x7f01800ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 -- 172.21.15.123:0/2487841464 >> v2:172.21.15.177:3300/0 conn(0x7f01800ace40 msgr2=0x7f01800ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 --2- 172.21.15.123:0/2487841464 >> v2:172.21.15.177:3300/0 conn(0x7f01800ace40 0x7f01800ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 -- 172.21.15.123:0/2487841464 >> v1:172.21.15.177:6789/0 conn(0x7f01800ab230 legacy=0x7f01800afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 -- 172.21.15.123:0/2487841464 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01800ac360 con 0x7f01800aae60 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f0184bda700 1 --2- 172.21.15.123:0/2487841464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800aae60 0x7f01800aec10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f01800ad800 tx=0x7f0174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9bc077a0830f6af server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f017f7fe700 1 -- 172.21.15.123:0/2487841464 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400b820 con 0x7f01800aae60 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f017f7fe700 1 -- 172.21.15.123:0/2487841464 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017400b980 con 0x7f01800aae60 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.817+0000 7f017f7fe700 1 -- 172.21.15.123:0/2487841464 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017400bd20 con 0x7f01800aae60 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 -- 172.21.15.123:0/2487841464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800aae60 msgr2=0x7f01800aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 --2- 172.21.15.123:0/2487841464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800aae60 0x7f01800aec10 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f01800ad800 tx=0x7f0174009250 comp rx=0 tx=0).stop 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 -- 172.21.15.123:0/2487841464 shutdown_connections 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 --2- 172.21.15.123:0/2487841464 >> v2:172.21.15.177:3300/0 conn(0x7f01800ace40 0x7f01800ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 --2- 172.21.15.123:0/2487841464 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800aae60 0x7f01800aec10 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 -- 172.21.15.123:0/2487841464 >> 172.21.15.123:0/2487841464 conn(0x7f0180097410 msgr2=0x7f0180067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 -- 172.21.15.123:0/2487841464 shutdown_connections 2023-10-06T04:33:06.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.818+0000 7f0186e3e700 1 -- 172.21.15.123:0/2487841464 wait complete. 2023-10-06T04:33:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.819+0000 7f0186e3e700 1 Processor -- start 2023-10-06T04:33:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.819+0000 7f0186e3e700 1 -- start start 2023-10-06T04:33:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.820+0000 7f0186e3e700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01800aae60 0x7f018006caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.820+0000 7f0186e3e700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01800ab230 0x7f0180099810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.820+0000 7f0186e3e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800ace40 0x7f018009aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.820+0000 7f0186e3e700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018006c610 con 0x7f01800ace40 2023-10-06T04:33:06.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.820+0000 7f0186e3e700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f018006ba00 con 0x7f01800aae60 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.820+0000 7f0186e3e700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f018006bce0 con 0x7f01800ab230 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.820+0000 7f01853db700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800ace40 0x7f018009aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800ace40 0x7f018009aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53410/0 (socket says 172.21.15.123:53410) 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f0184bda700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01800aae60 0x7f018006caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 -- 172.21.15.123:0/3083754998 learned_addr learned my addr 172.21.15.123:0/3083754998 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f017ffff700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01800ab230 0x7f0180099810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 -- 172.21.15.123:0/3083754998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01800ab230 msgr2=0x7f0180099810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01800ab230 0x7f0180099810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 -- 172.21.15.123:0/3083754998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01800aae60 msgr2=0x7f018006caa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:06.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01800aae60 0x7f018006caa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 -- 172.21.15.123:0/3083754998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01740094f0 con 0x7f01800ace40 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f01853db700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800ace40 0x7f018009aab0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0178009fa0 tx=0x7f0178009900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f017cff9700 1 -- 172.21.15.123:0/3083754998 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f017800ebc0 con 0x7f01800ace40 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f017cff9700 1 -- 172.21.15.123:0/3083754998 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f017800ed20 con 0x7f01800ace40 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f017cff9700 1 -- 172.21.15.123:0/3083754998 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0178014670 con 0x7f01800ace40 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f018006bfc0 con 0x7f01800ace40 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.821+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f018014e610 con 0x7f01800ace40 2023-10-06T04:33:06.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.822+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f018006c780 con 0x7f01800ace40 2023-10-06T04:33:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.823+0000 7f017cff9700 1 -- 172.21.15.123:0/3083754998 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f01780149e0 con 0x7f01800ace40 2023-10-06T04:33:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.823+0000 7f017cff9700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0164066f30 0x7f01640693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.823+0000 7f017cff9700 1 -- 172.21.15.123:0/3083754998 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f0178084d50 con 0x7f01800ace40 2023-10-06T04:33:06.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.823+0000 7f0184bda700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0164066f30 0x7f01640693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:06.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.823+0000 7f0184bda700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0164066f30 0x7f01640693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f01800ad800 tx=0x7f0174016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:06.828 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:06.827+0000 7f017cff9700 1 -- 172.21.15.123:0/3083754998 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0178088030 con 0x7f01800ace40 2023-10-06T04:33:07.011 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.010+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f018006d550 con 0x7f01800ace40 2023-10-06T04:33:07.012 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.011+0000 7f017cff9700 1 -- 172.21.15.123:0/3083754998 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17156 (secure 0 0 0) 0x7f0178051e00 con 0x7f01800ace40 2023-10-06T04:33:07.012 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:07.012 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:33:04.304393+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:33:07.014 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0164066f30 msgr2=0x7f01640693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0164066f30 0x7f01640693e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f01800ad800 tx=0x7f0174016000 comp rx=0 tx=0).stop 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800ace40 msgr2=0x7f018009aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800ace40 0x7f018009aab0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f0178009fa0 tx=0x7f0178009900 comp rx=0 tx=0).stop 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 shutdown_connections 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0164066f30 0x7f01640693e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f01800ace40 0x7f018009aab0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f01800aae60 0x7f018006caa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 --2- 172.21.15.123:0/3083754998 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f01800ab230 0x7f0180099810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 >> 172.21.15.123:0/3083754998 conn(0x7f0180097410 msgr2=0x7f01800b0f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 shutdown_connections 2023-10-06T04:33:07.015 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.013+0000 7f0186e3e700 1 -- 172.21.15.123:0/3083754998 wait complete. 2023-10-06T04:33:07.026 INFO:tasks.cephfs.filesystem:Destroying file system recovery_fs and related pools 2023-10-06T04:33:07.027 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:33:07.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.143+0000 7fc3cae12700 1 Processor -- start 2023-10-06T04:33:07.144 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.143+0000 7fc3cae12700 1 -- start start 2023-10-06T04:33:07.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3cae12700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c4067660 0x7fc3c4067a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3cae12700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc3c40c9570 0x7fc3c406e250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3cae12700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc3c4067f70 con 0x7fc3c40ca5b0 2023-10-06T04:33:07.147 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3cae12700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc3c406a750 con 0x7fc3c40c9570 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3cae12700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c406aa30 con 0x7fc3c4067660 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c93af700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc3c40ca5b0 0x7fc3c406e790 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60548/0 (socket says 172.21.15.123:60548) 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c93af700 1 -- 172.21.15.123:0/1445334216 learned_addr learned my addr 172.21.15.123:0/1445334216 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c8bae700 1 --2- 172.21.15.123:0/1445334216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c4067660 0x7fc3c4067a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c3fff700 1 --2- 172.21.15.123:0/1445334216 >> v2:172.21.15.177:3300/0 conn(0x7fc3c40c9570 0x7fc3c406e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c37fe700 1 -- 172.21.15.123:0/1445334216 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 636337387 0 0) 0x7fc3c4067f70 con 0x7fc3c40ca5b0 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c37fe700 1 -- 172.21.15.123:0/1445334216 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc3b4003570 con 0x7fc3c40ca5b0 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c37fe700 1 -- 172.21.15.123:0/1445334216 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 564928618 0 0) 0x7fc3b4003570 con 0x7fc3c40ca5b0 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c8bae700 1 -- 172.21.15.123:0/1445334216 >> v2:172.21.15.177:3300/0 conn(0x7fc3c40c9570 msgr2=0x7fc3c406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c8bae700 1 --2- 172.21.15.123:0/1445334216 >> v2:172.21.15.177:3300/0 conn(0x7fc3c40c9570 0x7fc3c406e250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c8bae700 1 -- 172.21.15.123:0/1445334216 >> v1:172.21.15.123:6789/0 conn(0x7fc3c40ca5b0 legacy=0x7fc3c406e790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.148 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c8bae700 1 -- 172.21.15.123:0/1445334216 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3c406ad10 con 0x7fc3c4067660 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.144+0000 7fc3c8bae700 1 --2- 172.21.15.123:0/1445334216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c4067660 0x7fc3c4067a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80098e0 tx=0x7fc3b80091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec707db768dd0235 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3c37fe700 1 -- 172.21.15.123:0/1445334216 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3bc0017f0 con 0x7fc3c4067660 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3c37fe700 1 -- 172.21.15.123:0/1445334216 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b8011420 con 0x7fc3c4067660 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3c37fe700 1 -- 172.21.15.123:0/1445334216 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b80117a0 con 0x7fc3c4067660 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 -- 172.21.15.123:0/1445334216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c4067660 msgr2=0x7fc3c4067a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 --2- 172.21.15.123:0/1445334216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c4067660 0x7fc3c4067a30 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc3b80098e0 tx=0x7fc3b80091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 -- 172.21.15.123:0/1445334216 shutdown_connections 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 --2- 172.21.15.123:0/1445334216 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c4067660 0x7fc3c4067a30 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 --2- 172.21.15.123:0/1445334216 >> v2:172.21.15.177:3300/0 conn(0x7fc3c40c9570 0x7fc3c406e250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 -- 172.21.15.123:0/1445334216 >> 172.21.15.123:0/1445334216 conn(0x7fc3c40cda80 msgr2=0x7fc3c40c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:07.149 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 -- 172.21.15.123:0/1445334216 shutdown_connections 2023-10-06T04:33:07.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.145+0000 7fc3cae12700 1 -- 172.21.15.123:0/1445334216 wait complete. 2023-10-06T04:33:07.150 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.146+0000 7fc3cae12700 1 Processor -- start 2023-10-06T04:33:07.152 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.146+0000 7fc3cae12700 1 -- start start 2023-10-06T04:33:07.152 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3cae12700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3c4067660 0x7fc3c41561b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.152 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3cae12700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3c40c9570 0x7fc3c4155b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3cae12700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c40ca5b0 0x7fc3c416cc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3cae12700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c40b0110 con 0x7fc3c4067660 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3cae12700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c416d4c0 con 0x7fc3c40c9570 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3cae12700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc3c416d7a0 con 0x7fc3c40ca5b0 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3c8bae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3c4067660 0x7fc3c41561b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3c8bae700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3c4067660 0x7fc3c41561b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53424/0 (socket says 172.21.15.123:53424) 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3c8bae700 1 -- 172.21.15.123:0/3111650347 learned_addr learned my addr 172.21.15.123:0/3111650347 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3c3fff700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3c40c9570 0x7fc3c4155b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.147+0000 7fc3c93af700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c40ca5b0 0x7fc3c416cc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.153 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c8bae700 1 -- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c40ca5b0 msgr2=0x7fc3c416cc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c8bae700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c40ca5b0 0x7fc3c416cc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c8bae700 1 -- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3c40c9570 msgr2=0x7fc3c4155b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c8bae700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3c40c9570 0x7fc3c4155b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c8bae700 1 -- 172.21.15.123:0/3111650347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc3b8009420 con 0x7fc3c4067660 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c8bae700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3c4067660 0x7fc3c41561b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc3c406ab70 tx=0x7fc3b8016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c0ff9700 1 -- 172.21.15.123:0/3111650347 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b801caf0 con 0x7fc3c4067660 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c0ff9700 1 -- 172.21.15.123:0/3111650347 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc3b801cc70 con 0x7fc3c4067660 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc3c416da80 con 0x7fc3c4067660 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3c0ff9700 1 -- 172.21.15.123:0/3111650347 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc3b801a450 con 0x7fc3c4067660 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.148+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc3c416d1b0 con 0x7fc3c4067660 2023-10-06T04:33:07.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.149+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc3c40cde80 con 0x7fc3c4067660 2023-10-06T04:33:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.149+0000 7fc3c0ff9700 1 -- 172.21.15.123:0/3111650347 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc3b801a820 con 0x7fc3c4067660 2023-10-06T04:33:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.150+0000 7fc3c0ff9700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc3a8066f30 0x7fc3a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.150+0000 7fc3c0ff9700 1 -- 172.21.15.123:0/3111650347 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fc3b8086180 con 0x7fc3c4067660 2023-10-06T04:33:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.150+0000 7fc3c3fff700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc3a8066f30 0x7fc3a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.150+0000 7fc3c3fff700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc3a8066f30 0x7fc3a80693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3c415f1f0 tx=0x7fc3b00091a0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.154+0000 7fc3c0ff9700 1 -- 172.21.15.123:0/3111650347 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc3b8053190 con 0x7fc3c4067660 2023-10-06T04:33:07.381 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.380+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc3c404bb80 con 0x7fc3c4067660 2023-10-06T04:33:07.382 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.381+0000 7fc3c0ff9700 1 -- 172.21.15.123:0/3111650347 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4609 (secure 0 0 0) 0x7fc3b80567b0 con 0x7fc3c4067660 2023-10-06T04:33:07.382 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:07.382 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:33:07.384 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc3a8066f30 msgr2=0x7fc3a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc3a8066f30 0x7fc3a80693e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fc3c415f1f0 tx=0x7fc3b00091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3c4067660 msgr2=0x7fc3c41561b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3c4067660 0x7fc3c41561b0 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7fc3c406ab70 tx=0x7fc3b8016000 comp rx=0 tx=0).stop 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 shutdown_connections 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc3a8066f30 0x7fc3a80693e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc3c4067660 0x7fc3c41561b0 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc3c40ca5b0 0x7fc3c416cc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 --2- 172.21.15.123:0/3111650347 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc3c40c9570 0x7fc3c4155b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 >> 172.21.15.123:0/3111650347 conn(0x7fc3c40cda80 msgr2=0x7fc3c40a9b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:07.385 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 shutdown_connections 2023-10-06T04:33:07.386 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.383+0000 7fc3cae12700 1 -- 172.21.15.123:0/3111650347 wait complete. 2023-10-06T04:33:07.386 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2023-10-06T04:33:07.396 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:33:07.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.512+0000 7f1e8d1da700 1 Processor -- start 2023-10-06T04:33:07.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.512+0000 7f1e8d1da700 1 -- start start 2023-10-06T04:33:07.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e8d1da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 0x7f1e880c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e8d1da700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8806af00 con 0x7f1e881597e0 2023-10-06T04:33:07.517 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e8d1da700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e8806b460 con 0x7f1e88159410 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e8d1da700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880c8050 con 0x7f1e88068d80 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e8659c700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f1e881597e0 0x7f1e88068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60552/0 (socket says 172.21.15.123:60552) 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e8659c700 1 -- 172.21.15.123:0/3554453348 learned_addr learned my addr 172.21.15.123:0/3554453348 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e8759e700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 0x7f1e880c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1817428178 0 0) 0x7f1e8806af00 con 0x7f1e881597e0 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.513+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e74003570 con 0x7f1e881597e0 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2882743964 0 0) 0x7f1e74003570 con 0x7f1e881597e0 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 msgr2=0x7f1e880c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 0x7f1e880c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 >> v1:172.21.15.177:6789/0 conn(0x7f1e88159410 legacy=0x7f1e880af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.518 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e8806b130 con 0x7f1e881597e0 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f1e78001470 con 0x7f1e881597e0 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 >> v1:172.21.15.123:6789/0 conn(0x7f1e881597e0 legacy=0x7f1e88068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 0x7f1e74003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e74003fd0 0x7f1e74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e74004d50 0x7f1e740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880c8050 con 0x7f1e74004d50 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7400bba0 con 0x7f1e74003fd0 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e7400df70 con 0x7f1e88068d80 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8759e700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e74004d50 0x7f1e740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e86d9d700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 0x7f1e74003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.519 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8659c700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e74003fd0 0x7f1e74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8759e700 1 -- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 msgr2=0x7f1e74003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8759e700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e88068d80 0x7f1e74003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8759e700 1 -- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e74003fd0 msgr2=0x7f1e74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8759e700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e74003fd0 0x7f1e74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8759e700 1 -- 172.21.15.123:0/3554453348 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e74003570 con 0x7f1e74004d50 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8659c700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e74003fd0 0x7f1e74004810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e8759e700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e74004d50 0x7f1e740051c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000e210 tx=0x7f1e8000b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b62b825cec5bf43 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8000eac0 con 0x7f1e74004d50 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e8000f040 con 0x7f1e74004d50 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.514+0000 7f1e85d9b700 1 -- 172.21.15.123:0/3554453348 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e8000ee70 con 0x7f1e74004d50 2023-10-06T04:33:07.520 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.515+0000 7f1e8d1da700 1 -- 172.21.15.123:0/3554453348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e74004d50 msgr2=0x7f1e740051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.515+0000 7f1e8d1da700 1 --2- 172.21.15.123:0/3554453348 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e74004d50 0x7f1e740051c0 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f1e8000e210 tx=0x7f1e8000b040 comp rx=0 tx=0).stop 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.515+0000 7f1e8759e700 1 -- 172.21.15.123:0/3554453348 reap_dead start 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.515+0000 7f1e8d1da700 1 -- 172.21.15.123:0/3554453348 shutdown_connections 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.515+0000 7f1e8d1da700 1 -- 172.21.15.123:0/3554453348 >> 172.21.15.123:0/3554453348 conn(0x7f1e880c9c90 msgr2=0x7f1e8806da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.515+0000 7f1e8d1da700 1 -- 172.21.15.123:0/3554453348 shutdown_connections 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.515+0000 7f1e8d1da700 1 -- 172.21.15.123:0/3554453348 wait complete. 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.516+0000 7f1e8d1da700 1 Processor -- start 2023-10-06T04:33:07.521 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.516+0000 7f1e8d1da700 1 -- start start 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8d1da700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88158510 0x7f1e88165840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8d1da700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e880c6b00 0x7f1e880c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8d1da700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e880a6f60 0x7f1e880a73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8d1da700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880a7920 con 0x7f1e88158510 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8d1da700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880c77e0 con 0x7f1e880c6b00 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8d1da700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e880a5c80 con 0x7f1e880a6f60 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88158510 0x7f1e88165840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88158510 0x7f1e88165840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53444/0 (socket says 172.21.15.123:53444) 2023-10-06T04:33:07.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 -- 172.21.15.123:0/1855331664 learned_addr learned my addr 172.21.15.123:0/1855331664 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8759e700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e880a6f60 0x7f1e880a73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 -- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e880a6f60 msgr2=0x7f1e880a73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e880a6f60 0x7f1e880a73e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 -- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e880c6b00 msgr2=0x7f1e880c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e8659c700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e880c6b00 0x7f1e880c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e880c6b00 0x7f1e880c3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.517+0000 7f1e86d9d700 1 -- 172.21.15.123:0/1855331664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e880c6f90 con 0x7f1e88158510 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.518+0000 7f1e86d9d700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88158510 0x7f1e88165840 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e70002ab0 tx=0x7f1e70007d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.518+0000 7f1e7effd700 1 -- 172.21.15.123:0/1855331664 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000f9a0 con 0x7f1e88158510 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.518+0000 7f1e7effd700 1 -- 172.21.15.123:0/1855331664 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e7000fb00 con 0x7f1e88158510 2023-10-06T04:33:07.525 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.518+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e880a5630 con 0x7f1e88158510 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.518+0000 7f1e7effd700 1 -- 172.21.15.123:0/1855331664 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e7000fec0 con 0x7f1e88158510 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.518+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e88165d80 con 0x7f1e88158510 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.519+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e880c7300 con 0x7f1e88158510 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.519+0000 7f1e7effd700 1 -- 172.21.15.123:0/1855331664 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1e70013600 con 0x7f1e88158510 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.519+0000 7f1e7effd700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.519+0000 7f1e7effd700 1 -- 172.21.15.123:0/1855331664 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7f1e70084d70 con 0x7f1e88158510 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.519+0000 7f1e8659c700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.520+0000 7f1e8659c700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e68066f30 0x7f1e680693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000e10 tx=0x7f1e8806b2c0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.524+0000 7f1e7effd700 1 -- 172.21.15.123:0/1855331664 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1e70051e20 con 0x7f1e88158510 2023-10-06T04:33:07.746 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.745+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f1e880a5490 con 0x7f1e88158510 2023-10-06T04:33:07.747 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.746+0000 7f1e7effd700 1 -- 172.21.15.123:0/1855331664 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) v1 ==== 94+0+4609 (secure 0 0 0) 0x7f1e880a5490 con 0x7f1e88158510 2023-10-06T04:33:07.747 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:07.747 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":28,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":26,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-10-06T04:32:37.297331+0000","modified":"2023-10-06T04:32:52.264088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5295},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5295":{"gid":5295,"name":"a","rank":0,"incarnation":23,"state":"up:active","state_seq":7,"addr":"172.21.15.123:6836/571369576","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":571369576},{"type":"v1","addr":"172.21.15.123:6836","nonce":571369576}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[6],"metadata_pool":7,"enabled":true,"fs_name":"recovery_fs","balancer":"","standby_count_wanted":0},"id":3}]} 2023-10-06T04:33:07.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.747+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e68066f30 msgr2=0x7f1e680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.747+0000 7f1e8d1da700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e68066f30 0x7f1e680693e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f1e78000e10 tx=0x7f1e8806b2c0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.747+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88158510 msgr2=0x7f1e88165840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.747+0000 7f1e8d1da700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88158510 0x7f1e88165840 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f1e70002ab0 tx=0x7f1e70007d10 comp rx=0 tx=0).stop 2023-10-06T04:33:07.749 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 shutdown_connections 2023-10-06T04:33:07.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1e68066f30 0x7f1e680693e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1e88158510 0x7f1e88165840 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1e880a6f60 0x7f1e880a73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 --2- 172.21.15.123:0/1855331664 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1e880c6b00 0x7f1e880c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 >> 172.21.15.123:0/1855331664 conn(0x7f1e880c9c90 msgr2=0x7f1e8806bb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:07.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 shutdown_connections 2023-10-06T04:33:07.750 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.748+0000 7f1e8d1da700 1 -- 172.21.15.123:0/1855331664 wait complete. 2023-10-06T04:33:07.751 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 28 2023-10-06T04:33:07.761 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:33:07.877 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.876+0000 7fdf5d666700 1 Processor -- start 2023-10-06T04:33:07.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.876+0000 7fdf5d666700 1 -- start start 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf5d666700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fdf581591b0 0x7fdf58159580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf5d666700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf580b05b0 0x7fdf580b3ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf5d666700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdf5815b370 con 0x7fdf580b0a20 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf5d666700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf5815b0d0 con 0x7fdf580b05b0 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf5d666700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdf580b0fe0 con 0x7fdf581591b0 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf580b05b0 0x7fdf580b3ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fdf580b05b0 0x7fdf580b3ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53446/0 (socket says 172.21.15.123:53446) 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 -- 172.21.15.123:0/4228007025 learned_addr learned my addr 172.21.15.123:0/4228007025 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:07.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 -- 172.21.15.123:0/4228007025 >> v2:172.21.15.177:3300/0 conn(0x7fdf581591b0 msgr2=0x7fdf58159580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 --2- 172.21.15.123:0/4228007025 >> v2:172.21.15.177:3300/0 conn(0x7fdf581591b0 0x7fdf58159580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 -- 172.21.15.123:0/4228007025 >> v1:172.21.15.177:6789/0 conn(0x7fdf580b0a20 legacy=0x7fdf580cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 -- 172.21.15.123:0/4228007025 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf580b12c0 con 0x7fdf580b05b0 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf567fc700 1 --2- 172.21.15.123:0/4228007025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf580b05b0 0x7fdf580b3ca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdf580889e0 tx=0x7fdf48009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c57c7584461fdf37 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf55ffb700 1 -- 172.21.15.123:0/4228007025 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800b820 con 0x7fdf580b05b0 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf55ffb700 1 -- 172.21.15.123:0/4228007025 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf4800b980 con 0x7fdf580b05b0 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.877+0000 7fdf55ffb700 1 -- 172.21.15.123:0/4228007025 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf4800bd20 con 0x7fdf580b05b0 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 -- 172.21.15.123:0/4228007025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf580b05b0 msgr2=0x7fdf580b3ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 --2- 172.21.15.123:0/4228007025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf580b05b0 0x7fdf580b3ca0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7fdf580889e0 tx=0x7fdf48009250 comp rx=0 tx=0).stop 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 -- 172.21.15.123:0/4228007025 shutdown_connections 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 --2- 172.21.15.123:0/4228007025 >> v2:172.21.15.177:3300/0 conn(0x7fdf581591b0 0x7fdf58159580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.881 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 --2- 172.21.15.123:0/4228007025 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf580b05b0 0x7fdf580b3ca0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 -- 172.21.15.123:0/4228007025 >> 172.21.15.123:0/4228007025 conn(0x7fdf580995a0 msgr2=0x7fdf58063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:07.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 -- 172.21.15.123:0/4228007025 shutdown_connections 2023-10-06T04:33:07.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.878+0000 7fdf5d666700 1 -- 172.21.15.123:0/4228007025 wait complete. 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.879+0000 7fdf5d666700 1 Processor -- start 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.879+0000 7fdf5d666700 1 -- start start 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf5d666700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf580b05b0 0x7fdf580a9e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf5d666700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf580b0a20 0x7fdf58169a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf5d666700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf581591b0 0x7fdf580adee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf5d666700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580aa3a0 con 0x7fdf581591b0 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf5d666700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580ab330 con 0x7fdf580b0a20 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf5d666700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdf580abe70 con 0x7fdf580b05b0 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf581591b0 0x7fdf580adee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf581591b0 0x7fdf580adee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53450/0 (socket says 172.21.15.123:53450) 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 -- 172.21.15.123:0/72819179 learned_addr learned my addr 172.21.15.123:0/72819179 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf567fc700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf580b0a20 0x7fdf58169a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 -- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf580b05b0 msgr2=0x7fdf580a9e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf56ffd700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf580b05b0 0x7fdf580a9e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf580b05b0 0x7fdf580a9e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 -- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf580b0a20 msgr2=0x7fdf58169a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf580b0a20 0x7fdf58169a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf577fe700 1 -- 172.21.15.123:0/72819179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdf480094f0 con 0x7fdf581591b0 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.880+0000 7fdf567fc700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf580b0a20 0x7fdf58169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.881+0000 7fdf577fe700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf581591b0 0x7fdf580adee0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000bbb0 tx=0x7fdf50009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.881+0000 7fdf3effd700 1 -- 172.21.15.123:0/72819179 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf500092e0 con 0x7fdf581591b0 2023-10-06T04:33:07.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.881+0000 7fdf3effd700 1 -- 172.21.15.123:0/72819179 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdf5000c7a0 con 0x7fdf581591b0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.881+0000 7fdf3effd700 1 -- 172.21.15.123:0/72819179 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdf5000cb40 con 0x7fdf581591b0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.881+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdf580ad260 con 0x7fdf581591b0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.881+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdf580ab820 con 0x7fdf581591b0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.882+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdf580b15a0 con 0x7fdf581591b0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.882+0000 7fdf3effd700 1 -- 172.21.15.123:0/72819179 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdf50012070 con 0x7fdf581591b0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.883+0000 7fdf3effd700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdf44066f60 0x7fdf44069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.883+0000 7fdf3effd700 1 -- 172.21.15.123:0/72819179 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fdf500843a0 con 0x7fdf581591b0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.883+0000 7fdf56ffd700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdf44066f60 0x7fdf44069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:07.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.883+0000 7fdf56ffd700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdf44066f60 0x7fdf44069410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006fd0 tx=0x7fdf4c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:07.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:07.887+0000 7fdf3effd700 1 -- 172.21.15.123:0/72819179 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdf50051450 con 0x7fdf581591b0 2023-10-06T04:33:08.069 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.068+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdf5804bb80 con 0x7fdf581591b0 2023-10-06T04:33:08.070 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.069+0000 7fdf3effd700 1 -- 172.21.15.123:0/72819179 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v43) v1 ==== 74+0+17156 (secure 0 0 0) 0x7fdf50054a70 con 0x7fdf581591b0 2023-10-06T04:33:08.070 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:08.071 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":43,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:33:04.304393+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}},{"pool":6,"pool_name":"cephfs_data","create_time":"2023-10-06T04:32:22.782310+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"41","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"recovery_fs"}}},{"pool":7,"pool_name":"recovery_fs_metadata","create_time":"2023-10-06T04:32:35.940980+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"42","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"recovery_fs"}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:33:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdf44066f60 msgr2=0x7fdf44069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdf44066f60 0x7fdf44069410 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fdf4c006fd0 tx=0x7fdf4c008040 comp rx=0 tx=0).stop 2023-10-06T04:33:08.072 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf581591b0 msgr2=0x7fdf580adee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf581591b0 0x7fdf580adee0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7fdf5000bbb0 tx=0x7fdf50009840 comp rx=0 tx=0).stop 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 shutdown_connections 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdf44066f60 0x7fdf44069410 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fdf581591b0 0x7fdf580adee0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fdf580b05b0 0x7fdf580a9e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 --2- 172.21.15.123:0/72819179 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fdf580b0a20 0x7fdf58169a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 >> 172.21.15.123:0/72819179 conn(0x7fdf580995a0 msgr2=0x7fdf580afd70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 shutdown_connections 2023-10-06T04:33:08.073 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.071+0000 7fdf5d666700 1 -- 172.21.15.123:0/72819179 wait complete. 2023-10-06T04:33:08.084 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail recovery_fs 2023-10-06T04:33:08.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.200+0000 7fd53499a700 1 Processor -- start 2023-10-06T04:33:08.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.200+0000 7fd53499a700 1 -- start start 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd53499a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd530157c70 0x7fd53015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd53499a700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd53006aca0 0x7fd530158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd53499a700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd53015b680 con 0x7fd530158580 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd53499a700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd530155270 con 0x7fd530157c70 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd53499a700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd53015b960 con 0x7fd53006aca0 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52e59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd530157c70 0x7fd53015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52ed9d700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fd530158580 0x7fd530068a20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:60564/0 (socket says 172.21.15.123:60564) 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52e59c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd530157c70 0x7fd53015ada0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53464/0 (socket says 172.21.15.123:53464) 2023-10-06T04:33:08.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52ed9d700 1 -- 172.21.15.123:0/192869889 learned_addr learned my addr 172.21.15.123:0/192869889 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52dd9b700 1 --2- 172.21.15.123:0/192869889 >> v2:172.21.15.177:3300/0 conn(0x7fd53006aca0 0x7fd530158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52e59c700 1 -- 172.21.15.123:0/192869889 >> v2:172.21.15.177:3300/0 conn(0x7fd53006aca0 msgr2=0x7fd530158040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52e59c700 1 --2- 172.21.15.123:0/192869889 >> v2:172.21.15.177:3300/0 conn(0x7fd53006aca0 0x7fd530158040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52e59c700 1 -- 172.21.15.123:0/192869889 >> v1:172.21.15.123:6789/0 conn(0x7fd530158580 legacy=0x7fd530068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52e59c700 1 -- 172.21.15.123:0/192869889 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd53015bc40 con 0x7fd530157c70 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52d59a700 1 -- 172.21.15.123:0/192869889 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1012837508 0 0) 0x7fd53015b680 con 0x7fd530158580 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52e59c700 1 --2- 172.21.15.123:0/192869889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd53015ada0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd530069830 tx=0x7fd5240091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60e7bdde192727d1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52d59a700 1 -- 172.21.15.123:0/192869889 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400b6b0 con 0x7fd530157c70 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52d59a700 1 -- 172.21.15.123:0/192869889 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52400b810 con 0x7fd530157c70 2023-10-06T04:33:08.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.202+0000 7fd52d59a700 1 -- 172.21.15.123:0/192869889 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52400bbb0 con 0x7fd530157c70 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 -- 172.21.15.123:0/192869889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 msgr2=0x7fd53015ada0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 --2- 172.21.15.123:0/192869889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd53015ada0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7fd530069830 tx=0x7fd5240091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 -- 172.21.15.123:0/192869889 shutdown_connections 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 --2- 172.21.15.123:0/192869889 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd53015ada0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 --2- 172.21.15.123:0/192869889 >> v2:172.21.15.177:3300/0 conn(0x7fd53006aca0 0x7fd530158040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 -- 172.21.15.123:0/192869889 >> 172.21.15.123:0/192869889 conn(0x7fd5300cd010 msgr2=0x7fd5300cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 -- 172.21.15.123:0/192869889 shutdown_connections 2023-10-06T04:33:08.207 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.203+0000 7fd53499a700 1 -- 172.21.15.123:0/192869889 wait complete. 2023-10-06T04:33:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.205+0000 7fd53499a700 1 Processor -- start 2023-10-06T04:33:08.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.205+0000 7fd53499a700 1 -- start start 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.205+0000 7fd53499a700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd53006aca0 0x7fd5300b29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.205+0000 7fd53499a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd5300aff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd53499a700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd530158580 0x7fd53016a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd53499a700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c2060 con 0x7fd530157c70 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd53499a700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c1730 con 0x7fd530158580 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd53499a700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5300c1a10 con 0x7fd53006aca0 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd5300aff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd5300aff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53468/0 (socket says 172.21.15.123:53468) 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 -- 172.21.15.123:0/1320378936 learned_addr learned my addr 172.21.15.123:0/1320378936 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52e59c700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd53006aca0 0x7fd5300b29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52ed9d700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd530158580 0x7fd53016a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.211 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 -- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd53006aca0 msgr2=0x7fd5300b29c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd53006aca0 0x7fd5300b29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 -- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd530158580 msgr2=0x7fd53016a500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd530158580 0x7fd53016a500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 -- 172.21.15.123:0/1320378936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd524009420 con 0x7fd530157c70 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52ed9d700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd530158580 0x7fd53016a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd52dd9b700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd5300aff50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd52000d890 tx=0x7fd520008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd51e7fc700 1 -- 172.21.15.123:0/1320378936 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000b600 con 0x7fd530157c70 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd51e7fc700 1 -- 172.21.15.123:0/1320378936 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd52000b760 con 0x7fd530157c70 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5300c1cf0 con 0x7fd530157c70 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd51e7fc700 1 -- 172.21.15.123:0/1320378936 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd52000bae0 con 0x7fd530157c70 2023-10-06T04:33:08.212 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.206+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5300b0780 con 0x7fd530157c70 2023-10-06T04:33:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.207+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5300c21d0 con 0x7fd530157c70 2023-10-06T04:33:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.208+0000 7fd51e7fc700 1 -- 172.21.15.123:0/1320378936 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd52001e460 con 0x7fd530157c70 2023-10-06T04:33:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.208+0000 7fd51e7fc700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd510066f30 0x7fd5100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.208+0000 7fd51e7fc700 1 -- 172.21.15.123:0/1320378936 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6159+0+0 (secure 0 0 0) 0x7fd520084d80 con 0x7fd530157c70 2023-10-06T04:33:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.208+0000 7fd52e59c700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd510066f30 0x7fd5100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.208+0000 7fd52e59c700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd510066f30 0x7fd5100693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd53006b670 tx=0x7fd524017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:08.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.212+0000 7fd51e7fc700 1 -- 172.21.15.123:0/1320378936 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd520051e30 con 0x7fd530157c70 2023-10-06T04:33:08.437 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.436+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "recovery_fs"} v 0) v1 -- 0x7fd53006a4a0 con 0x7fd530157c70 2023-10-06T04:33:08.438 INFO:tasks.ceph.mon.a.smithi123.stderr:2023-10-06T04:33:08.437+0000 7fea90960700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-10-06T04:33:08.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.440+0000 7fd51e7fc700 1 -- 172.21.15.123:0/1320378936 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "recovery_fs"}]=0 recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v29) v1 ==== 171+0+0 (secure 0 0 0) 0x7fd520055450 con 0x7fd530157c70 2023-10-06T04:33:08.442 INFO:tasks.ceph.mds.a.smithi123.stderr: -3428> 2023-10-06T04:32:49.255+0000 7f99813f7700 -1 mds.0.openfiles _load_finish got (2) No such file or directory 2023-10-06T04:33:08.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd510066f30 msgr2=0x7fd5100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd510066f30 0x7fd5100693e0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fd53006b670 tx=0x7fd524017000 comp rx=0 tx=0).stop 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 msgr2=0x7fd5300aff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd5300aff50 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fd52000d890 tx=0x7fd520008040 comp rx=0 tx=0).stop 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 shutdown_connections 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd510066f30 0x7fd5100693e0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd530158580 0x7fd53016a500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd53006aca0 0x7fd5300b29c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 --2- 172.21.15.123:0/1320378936 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd530157c70 0x7fd5300aff50 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 >> 172.21.15.123:0/1320378936 conn(0x7fd5300cd010 msgr2=0x7fd53006efa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 shutdown_connections 2023-10-06T04:33:08.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.443+0000 7fd53499a700 1 -- 172.21.15.123:0/1320378936 wait complete. 2023-10-06T04:33:08.446 INFO:tasks.ceph.mds.a.smithi123.stderr: -1828> 2023-10-06T04:32:54.288+0000 7f99813f7700 -1 log_channel(cluster) log [ERR] : bad backtrace on directory inode 0x10000000000 2023-10-06T04:33:08.446 INFO:tasks.ceph.mds.a.smithi123.stderr: -1635> 2023-10-06T04:32:54.288+0000 7f99813f7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000000 [...2,head] /subdir/ auth v1 pv4 ap=3 DIRTYPARENT f(v0 1=1+0) n() (inest lock dirty) (ifile lock->sync w=1 flushing) (iversion lock) | dirtyscattered=2 lock=1 dirfrag=1 dirtyparent=1 waiter=1 authpin=1 scrubqueue=0 0x5584fbdff080]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":true,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x10000000000:[<0x1/subdir v1>]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=1+0)","ondisk_value.rstat":"n(v0 1=0+1)","memory_value.dirstat":"f(v0 1=1+0)","memory_value.rstat":"n()","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T04:33:08.446 INFO:tasks.ceph.mds.a.smithi123.stderr: -1617> 2023-10-06T04:32:54.289+0000 7f99813f7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x10000000001 [2,head] /subdir/sixmegs auth v1 ap=1 DIRTYPARENT s=6291456 n() (iversion lock) | dirtyparent=1 authpin=1 scrubqueue=0 0x5584fbe6b600]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":0,"ondisk_value":"(6)0x10000000001:[<0x10000000000/sixmegs v8>,<0x1/subdir v10>]//[]","memoryvalue":"(6)0x10000000001:[<0x10000000000/sixmegs v1>,<0x1/subdir v1>]//[]","error_str":"On-disk backtrace is divergent or newer"},"raw_stats":{"checked":false,"passed":false,"read_ret_val":0,"ondisk_value.dirstat":"f()","ondisk_value.rstat":"n()","memory_value.dirstat":"f()","memory_value.rstat":"n()","error_str":""},"return_code":0} 2023-10-06T04:33:08.447 INFO:teuthology.orchestra.run.smithi123.stderr:recovery_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-10-06T04:33:08.447 INFO:tasks.ceph.mds.a.smithi123.stderr: -1377> 2023-10-06T04:32:54.293+0000 7f99813f7700 -1 mds.0.scrubstack _validate_inode_done scrub error on inode [inode 0x1 [...2,head] / auth v4 ap=5 snaprealm=0x5584fb053200 DIRTYPARENT f(v1 1=0+1) n(v0 1=0+1)/n() (idft sync r=1) (inest sync r=1) (ifile sync->mix r=1) (iversion lock) | dirtyscattered=0 lock=3 dirfrag=1 openingsnapparents=0 dirtyparent=1 dirty=1 waiter=1 authpin=1 scrubqueue=0 0x5584fbdfe580]: {"performed_validation":true,"passed_validation":false,"backtrace":{"checked":true,"passed":false,"read_ret_val":-61,"ondisk_value":"(-1)0x0:[]//[]","memoryvalue":"(7)0x1:[]//[]","error_str":"failed to read off disk; see retval"},"raw_stats":{"checked":true,"passed":true,"read_ret_val":0,"ondisk_value.dirstat":"f(v0 1=0+1)","ondisk_value.rstat":"n(v0 2=0+2)","memory_value.dirstat":"f(v1 1=0+1)","memory_value.rstat":"n(v0 1=0+1)","error_str":"freshly-calculated rstats don't match existing ones (will be fixed)"},"return_code":-61} 2023-10-06T04:33:08.447 INFO:tasks.ceph.mds.a.smithi123.stderr: -1156> 2023-10-06T04:32:54.588+0000 7f9984bfe700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-10-06T04:33:08.459 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm recovery_fs --yes-i-really-mean-it 2023-10-06T04:33:08.469 INFO:tasks.ceph.mds.a.smithi123.stdout:starting mds.a at 2023-10-06T04:33:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.577+0000 7f1abc430700 1 Processor -- start 2023-10-06T04:33:08.579 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.577+0000 7f1abc430700 1 -- start start 2023-10-06T04:33:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1abc430700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ab4157430 0x7f1ab40ca9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1abc430700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1ab40caf00 0x7f1ab40c8a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.581 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1abc430700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ab40c9d70 con 0x7f1ab40c8f80 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1abc430700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab4157800 con 0x7f1ab4157430 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1abc430700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ab4159200 con 0x7f1ab40caf00 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1aba1cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ab4157430 0x7f1ab40ca9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1aba1cc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f1ab4157430 0x7f1ab40ca9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53490/0 (socket says 172.21.15.123:53490) 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.578+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1538989478 learned_addr learned my addr 172.21.15.123:0/1538989478 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1ab99cb700 1 --2- 172.21.15.123:0/1538989478 >> v2:172.21.15.177:3300/0 conn(0x7f1ab40caf00 0x7f1ab40c8a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1538989478 >> v2:172.21.15.177:3300/0 conn(0x7f1ab40caf00 msgr2=0x7f1ab40c8a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1aba1cc700 1 --2- 172.21.15.123:0/1538989478 >> v2:172.21.15.177:3300/0 conn(0x7f1ab40caf00 0x7f1ab40c8a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1538989478 >> v1:172.21.15.177:6789/0 conn(0x7f1ab40c8f80 legacy=0x7f1ab415af10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:08.582 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1538989478 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ab40cb3a0 con 0x7f1ab4157430 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1aba1cc700 1 --2- 172.21.15.123:0/1538989478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4157430 0x7f1ab40ca9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406f470 tx=0x7f1aa4009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9af74f0dd8f36e8c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1ab91ca700 1 -- 172.21.15.123:0/1538989478 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4005c10 con 0x7f1ab4157430 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1ab91ca700 1 -- 172.21.15.123:0/1538989478 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4005d70 con 0x7f1ab4157430 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1ab91ca700 1 -- 172.21.15.123:0/1538989478 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa400b840 con 0x7f1ab4157430 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1abc430700 1 -- 172.21.15.123:0/1538989478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4157430 msgr2=0x7f1ab40ca9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.579+0000 7f1abc430700 1 --2- 172.21.15.123:0/1538989478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4157430 0x7f1ab40ca9c0 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f1ab406f470 tx=0x7f1aa4009220 comp rx=0 tx=0).stop 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.580+0000 7f1abc430700 1 -- 172.21.15.123:0/1538989478 shutdown_connections 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.580+0000 7f1abc430700 1 --2- 172.21.15.123:0/1538989478 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab4157430 0x7f1ab40ca9c0 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.580+0000 7f1abc430700 1 --2- 172.21.15.123:0/1538989478 >> v2:172.21.15.177:3300/0 conn(0x7f1ab40caf00 0x7f1ab40c8a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.583 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.580+0000 7f1abc430700 1 -- 172.21.15.123:0/1538989478 >> 172.21.15.123:0/1538989478 conn(0x7f1ab4097450 msgr2=0x7f1ab4067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:08.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.580+0000 7f1abc430700 1 -- 172.21.15.123:0/1538989478 shutdown_connections 2023-10-06T04:33:08.584 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.580+0000 7f1abc430700 1 -- 172.21.15.123:0/1538989478 wait complete. 2023-10-06T04:33:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.581+0000 7f1abc430700 1 Processor -- start 2023-10-06T04:33:08.586 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.581+0000 7f1abc430700 1 -- start start 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab40c8f80 0x7f1ab4168680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab40caf00 0x7f1ab40c5de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ab4157430 0x7f1ab40cbc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40c7780 con 0x7f1ab40c8f80 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40c66e0 con 0x7f1ab40caf00 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ab40c69c0 con 0x7f1ab4157430 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab40c8f80 0x7f1ab4168680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab40c8f80 0x7f1ab4168680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53502/0 (socket says 172.21.15.123:53502) 2023-10-06T04:33:08.587 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1815319621 learned_addr learned my addr 172.21.15.123:0/1815319621 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1ab99cb700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab40caf00 0x7f1ab40c5de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba9cd700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ab4157430 0x7f1ab40cbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ab4157430 msgr2=0x7f1ab40cbc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ab4157430 0x7f1ab40cbc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab40caf00 msgr2=0x7f1ab40c5de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab40caf00 0x7f1ab40c5de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 -- 172.21.15.123:0/1815319621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1aa4009480 con 0x7f1ab40c8f80 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba9cd700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ab4157430 0x7f1ab40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aba1cc700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab40c8f80 0x7f1ab4168680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40030c0 tx=0x7f1aa4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:08.588 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aaa7fc700 1 -- 172.21.15.123:0/1815319621 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa4011a70 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aaa7fc700 1 -- 172.21.15.123:0/1815319621 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1aa4011bf0 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ab40c6ca0 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1aaa7fc700 1 -- 172.21.15.123:0/1815319621 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1aa401c420 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.582+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ab40c7260 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.583+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ab406d840 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.584+0000 7f1aaa7fc700 1 -- 172.21.15.123:0/1815319621 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f1aa401c790 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.584+0000 7f1aaa7fc700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a98066f30 0x7f1a980693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.584+0000 7f1aaa7fc700 1 -- 172.21.15.123:0/1815319621 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f1aa4085de0 con 0x7f1ab40c8f80 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.584+0000 7f1ab99cb700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a98066f30 0x7f1a980693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:08.589 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.584+0000 7f1ab99cb700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a98066f30 0x7f1a980693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40c55c0 tx=0x7f1ab0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:08.590 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.588+0000 7f1aaa7fc700 1 -- 172.21.15.123:0/1815319621 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f1aa4052e30 con 0x7f1ab40c8f80 2023-10-06T04:33:08.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:08.815+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ab40c9d70 con 0x7f1ab40c8f80 2023-10-06T04:33:09.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.442+0000 7f1aaa7fc700 1 -- 172.21.15.123:0/1815319621 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "recovery_fs", "yes_i_really_mean_it": true}]=0 v30) v1 ==== 109+0+0 (secure 0 0 0) 0x7f1ab40c9d70 con 0x7f1ab40c8f80 2023-10-06T04:33:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a98066f30 msgr2=0x7f1a980693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:09.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a98066f30 0x7f1a980693e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1ab40c55c0 tx=0x7f1ab0008040 comp rx=0 tx=0).stop 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab40c8f80 msgr2=0x7f1ab4168680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab40c8f80 0x7f1ab4168680 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f1aa40030c0 tx=0x7f1aa4015000 comp rx=0 tx=0).stop 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 shutdown_connections 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1a98066f30 0x7f1a980693e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f1ab40c8f80 0x7f1ab4168680 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1ab4157430 0x7f1ab40cbc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 --2- 172.21.15.123:0/1815319621 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1ab40caf00 0x7f1ab40c5de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 >> 172.21.15.123:0/1815319621 conn(0x7f1ab4097450 msgr2=0x7f1ab4153350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 shutdown_connections 2023-10-06T04:33:09.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.445+0000 7f1abc430700 1 -- 172.21.15.123:0/1815319621 wait complete. 2023-10-06T04:33:09.463 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm recovery_fs_metadata recovery_fs_metadata --yes-i-really-really-mean-it 2023-10-06T04:33:09.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.626+0000 7f1728211700 1 Processor -- start 2023-10-06T04:33:09.629 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.626+0000 7f1728211700 1 -- start start 2023-10-06T04:33:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1728211700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17200678b0 0x7f17200aec90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1728211700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f1720067c80 0x7f17200af1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:09.631 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1728211700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f1720159280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1728211700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17200af710 con 0x7f17200678b0 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1728211700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f172006add0 con 0x7f1720067c80 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1728211700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1720068570 con 0x7f1720158e30 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1725fad700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17200678b0 0x7f17200aec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1725fad700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f17200678b0 0x7f17200aec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53512/0 (socket says 172.21.15.123:53512) 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.627+0000 7f1725fad700 1 -- 172.21.15.123:0/1980021060 learned_addr learned my addr 172.21.15.123:0/1980021060 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f17267ae700 1 --2- 172.21.15.123:0/1980021060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f1720159280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f17257ac700 1 --2- 172.21.15.123:0/1980021060 >> v2:172.21.15.177:3300/0 conn(0x7f1720067c80 0x7f17200af1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1725fad700 1 -- 172.21.15.123:0/1980021060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 msgr2=0x7f1720159280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1725fad700 1 --2- 172.21.15.123:0/1980021060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f1720159280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.632 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1725fad700 1 -- 172.21.15.123:0/1980021060 >> v2:172.21.15.177:3300/0 conn(0x7f1720067c80 msgr2=0x7f17200af1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1725fad700 1 --2- 172.21.15.123:0/1980021060 >> v2:172.21.15.177:3300/0 conn(0x7f1720067c80 0x7f17200af1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1725fad700 1 -- 172.21.15.123:0/1980021060 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17200af930 con 0x7f17200678b0 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1725fad700 1 --2- 172.21.15.123:0/1980021060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f17200aec90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f172006dec0 tx=0x7f1710009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338e30ea2a2299d2 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1724fab700 1 -- 172.21.15.123:0/1980021060 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171000b860 con 0x7f17200678b0 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1724fab700 1 -- 172.21.15.123:0/1980021060 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171000b9c0 con 0x7f17200678b0 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1724fab700 1 -- 172.21.15.123:0/1980021060 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171000bd60 con 0x7f17200678b0 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1728211700 1 -- 172.21.15.123:0/1980021060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 msgr2=0x7f17200aec90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.628+0000 7f1728211700 1 --2- 172.21.15.123:0/1980021060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f17200aec90 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f172006dec0 tx=0x7f1710009250 comp rx=0 tx=0).stop 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.629+0000 7f1728211700 1 -- 172.21.15.123:0/1980021060 shutdown_connections 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.629+0000 7f1728211700 1 --2- 172.21.15.123:0/1980021060 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f1720159280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.629+0000 7f1728211700 1 --2- 172.21.15.123:0/1980021060 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f17200aec90 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.629+0000 7f1728211700 1 --2- 172.21.15.123:0/1980021060 >> v2:172.21.15.177:3300/0 conn(0x7f1720067c80 0x7f17200af1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.629+0000 7f1728211700 1 -- 172.21.15.123:0/1980021060 >> 172.21.15.123:0/1980021060 conn(0x7f17200cdb10 msgr2=0x7f17200abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:09.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.629+0000 7f1728211700 1 -- 172.21.15.123:0/1980021060 shutdown_connections 2023-10-06T04:33:09.634 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.629+0000 7f1728211700 1 -- 172.21.15.123:0/1980021060 wait complete. 2023-10-06T04:33:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.630+0000 7f1728211700 1 Processor -- start 2023-10-06T04:33:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.630+0000 7f1728211700 1 -- start start 2023-10-06T04:33:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.630+0000 7f1728211700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f172014c920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:09.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.630+0000 7f1728211700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1720067c80 0x7f172014f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1728211700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f172014fc70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1728211700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172015f210 con 0x7f17200678b0 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1728211700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f172014dc70 con 0x7f1720067c80 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1728211700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f172015ec50 con 0x7f1720158e30 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f172014c920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f172014c920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:53516/0 (socket says 172.21.15.123:53516) 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 -- 172.21.15.123:0/2056240361 learned_addr learned my addr 172.21.15.123:0/2056240361 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f17267ae700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f172014fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f17257ac700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1720067c80 0x7f172014f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 -- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 msgr2=0x7f172014fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f172014fc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.637 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 -- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1720067c80 msgr2=0x7f172014f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1720067c80 0x7f172014f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 -- 172.21.15.123:0/2056240361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17100094f0 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1725fad700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f172014c920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f172006a1a0 tx=0x7f1710017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1715ffb700 1 -- 172.21.15.123:0/2056240361 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171000bcd0 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1715ffb700 1 -- 172.21.15.123:0/2056240361 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f171001a630 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f172015ef30 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1715ffb700 1 -- 172.21.15.123:0/2056240361 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f171001a9f0 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.631+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f172014ce60 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.632+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f172014da40 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.633+0000 7f1715ffb700 1 -- 172.21.15.123:0/2056240361 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f17100233e0 con 0x7f17200678b0 2023-10-06T04:33:09.638 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.633+0000 7f1715ffb700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1704066f30 0x7f17040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:09.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.633+0000 7f1715ffb700 1 -- 172.21.15.123:0/2056240361 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6245+0+0 (secure 0 0 0) 0x7f1710086460 con 0x7f17200678b0 2023-10-06T04:33:09.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.633+0000 7f17257ac700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1704066f30 0x7f17040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:09.639 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.634+0000 7f17257ac700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1704066f30 0x7f17040693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1720153af0 tx=0x7f171c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:09.640 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.638+0000 7f1715ffb700 1 -- 172.21.15.123:0/2056240361 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f172014da40 con 0x7f17200678b0 2023-10-06T04:33:09.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:09.819+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f172014da40 con 0x7f17200678b0 2023-10-06T04:33:10.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.445+0000 7f1715ffb700 1 -- 172.21.15.123:0/2056240361 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "recovery_fs_metadata", "pool2": "recovery_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'recovery_fs_metadata' removed v45) v1 ==== 196+0+0 (secure 0 0 0) 0x7f172014da40 con 0x7f17200678b0 2023-10-06T04:33:10.451 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1704066f30 msgr2=0x7f17040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1704066f30 0x7f17040693e0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f1720153af0 tx=0x7f171c008040 comp rx=0 tx=0).stop 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 msgr2=0x7f172014c920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f172014c920 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f172006a1a0 tx=0x7f1710017000 comp rx=0 tx=0).stop 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 shutdown_connections 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f1704066f30 0x7f17040693e0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f17200678b0 0x7f172014c920 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f1720158e30 0x7f172014fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 --2- 172.21.15.123:0/2056240361 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f1720067c80 0x7f172014f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 >> 172.21.15.123:0/2056240361 conn(0x7f17200cdb10 msgr2=0x7f172006b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:10.452 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 shutdown_connections 2023-10-06T04:33:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.449+0000 7f1728211700 1 -- 172.21.15.123:0/2056240361 wait complete. 2023-10-06T04:33:10.453 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'recovery_fs_metadata' removed 2023-10-06T04:33:10.467 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-10-06T04:33:10.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.590+0000 7f3b6accb700 1 Processor -- start 2023-10-06T04:33:10.593 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.591+0000 7f3b6accb700 1 -- start start 2023-10-06T04:33:10.596 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.591+0000 7f3b6accb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 0x7f3b64068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.591+0000 7f3b6accb700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b640685e0 con 0x7f3b640cc290 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.591+0000 7f3b6accb700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b640632c0 con 0x7f3b6406a150 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.591+0000 7f3b6accb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64068fb0 con 0x7f3b64159610 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.591+0000 7f3b63fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3b640cc290 0x7f3b64069a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55366/0 (socket says 172.21.15.123:55366) 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b63fff700 1 -- 172.21.15.123:0/3013094999 learned_addr learned my addr 172.21.15.123:0/3013094999 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b68a67700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 0x7f3b64068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3238655412 0 0) 0x7f3b640685e0 con 0x7f3b640cc290 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3b54003570 con 0x7f3b640cc290 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2924820159 0 0) 0x7f3b54003570 con 0x7f3b640cc290 2023-10-06T04:33:10.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 msgr2=0x7f3b64068a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 0x7f3b64068a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 >> v1:172.21.15.177:6789/0 conn(0x7f3b6406a150 legacy=0x7f3b640c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640679d0 con 0x7f3b640cc290 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f3b500015f0 con 0x7f3b640cc290 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 >> v1:172.21.15.123:6789/0 conn(0x7f3b640cc290 legacy=0x7f3b64069a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 0x7f3b54003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b54004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b54004d50 0x7f3b540051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b64068fb0 con 0x7f3b54004010 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5400bbf0 con 0x7f3b54004d50 2023-10-06T04:33:10.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b5400bdb0 con 0x7f3b64159610 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b63fff700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b54004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b68a67700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 0x7f3b54003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b69268700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b54004d50 0x7f3b540051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b63fff700 1 -- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 msgr2=0x7f3b54003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b63fff700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b64159610 0x7f3b54003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b63fff700 1 -- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b54004d50 msgr2=0x7f3b540051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b63fff700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b54004d50 0x7f3b540051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.592+0000 7f3b63fff700 1 -- 172.21.15.123:0/3013094999 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b54003570 con 0x7f3b54004010 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b63fff700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b54004810 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000c390 tx=0x7f3b50007920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b6afa64ab0e6df9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5000cc60 con 0x7f3b54004010 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5000cde0 con 0x7f3b54004010 2023-10-06T04:33:10.599 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b637fe700 1 -- 172.21.15.123:0/3013094999 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b50007320 con 0x7f3b54004010 2023-10-06T04:33:10.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b6accb700 1 -- 172.21.15.123:0/3013094999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 msgr2=0x7f3b54004810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:10.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b6accb700 1 --2- 172.21.15.123:0/3013094999 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b54004810 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7f3b5000c390 tx=0x7f3b50007920 comp rx=0 tx=0).stop 2023-10-06T04:33:10.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b69268700 1 -- 172.21.15.123:0/3013094999 reap_dead start 2023-10-06T04:33:10.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b6accb700 1 -- 172.21.15.123:0/3013094999 shutdown_connections 2023-10-06T04:33:10.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b6accb700 1 -- 172.21.15.123:0/3013094999 >> 172.21.15.123:0/3013094999 conn(0x7f3b640cd250 msgr2=0x7f3b640c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:10.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b6accb700 1 -- 172.21.15.123:0/3013094999 shutdown_connections 2023-10-06T04:33:10.600 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.593+0000 7f3b6accb700 1 -- 172.21.15.123:0/3013094999 wait complete. 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.595+0000 7f3b6accb700 1 Processor -- start 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.595+0000 7f3b6accb700 1 -- start start 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b6accb700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b64169e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b6accb700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b641481f0 0x7f3b640c8870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b6accb700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b640c8db0 0x7f3b640c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b6accb700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640c66d0 con 0x7f3b54004010 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b6accb700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640c74f0 con 0x7f3b640c8db0 2023-10-06T04:33:10.603 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b6accb700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b640c77d0 con 0x7f3b641481f0 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b64169e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b64169e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41746/0 (socket says 172.21.15.123:41746) 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 -- 172.21.15.123:0/4273506187 learned_addr learned my addr 172.21.15.123:0/4273506187 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 -- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b641481f0 msgr2=0x7f3b640c8870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b63fff700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b641481f0 0x7f3b640c8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b69268700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b640c8db0 0x7f3b640c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b641481f0 0x7f3b640c8870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 -- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b640c8db0 msgr2=0x7f3b640c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b640c8db0 0x7f3b640c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 -- 172.21.15.123:0/4273506187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b640c7ab0 con 0x7f3b54004010 2023-10-06T04:33:10.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b63fff700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b641481f0 0x7f3b640c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b69268700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b640c8db0 0x7f3b640c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.596+0000 7f3b68a67700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b64169e20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3b58011d40 tx=0x7f3b580091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.597+0000 7f3b60ff9700 1 -- 172.21.15.123:0/4273506187 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b5800fac0 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.597+0000 7f3b60ff9700 1 -- 172.21.15.123:0/4273506187 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b5800fc20 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.597+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b640c7d90 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.597+0000 7f3b60ff9700 1 -- 172.21.15.123:0/4273506187 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b58014440 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.597+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b640c8350 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.598+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b640c8570 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.598+0000 7f3b60ff9700 1 -- 172.21.15.123:0/4273506187 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f3b58014a80 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.598+0000 7f3b60ff9700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3b48066f30 0x7f3b480693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.598+0000 7f3b60ff9700 1 -- 172.21.15.123:0/4273506187 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 5797+0+0 (secure 0 0 0) 0x7f3b58018030 con 0x7f3b54004010 2023-10-06T04:33:10.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.598+0000 7f3b63fff700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3b48066f30 0x7f3b480693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:10.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.599+0000 7f3b63fff700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3b48066f30 0x7f3b480693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3b640c2550 tx=0x7f3b50015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:10.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.602+0000 7f3b60ff9700 1 -- 172.21.15.123:0/4273506187 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f3b58052520 con 0x7f3b54004010 2023-10-06T04:33:10.786 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:10.784+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3b64067cb0 con 0x7f3b54004010 2023-10-06T04:33:11.456 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.454+0000 7f3b60ff9700 1 -- 172.21.15.123:0/4273506187 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v46) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3b5805b030 con 0x7f3b54004010 2023-10-06T04:33:11.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3b48066f30 msgr2=0x7f3b480693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.460 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3b48066f30 0x7f3b480693e0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3b640c2550 tx=0x7f3b50015000 comp rx=0 tx=0).stop 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 msgr2=0x7f3b64169e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b64169e20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7f3b58011d40 tx=0x7f3b580091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 shutdown_connections 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3b48066f30 0x7f3b480693e0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3b54004010 0x7f3b64169e20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3b640c8db0 0x7f3b640c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 --2- 172.21.15.123:0/4273506187 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3b641481f0 0x7f3b640c8870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 >> 172.21.15.123:0/4273506187 conn(0x7f3b640cd250 msgr2=0x7f3b640c2aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 shutdown_connections 2023-10-06T04:33:11.461 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.458+0000 7f3b6accb700 1 -- 172.21.15.123:0/4273506187 wait complete. 2023-10-06T04:33:11.462 INFO:teuthology.orchestra.run.smithi123.stderr:pool 'cephfs_data' removed 2023-10-06T04:33:11.473 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mds debug_mds 2023-10-06T04:33:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.599+0000 7fb94f4ea700 1 Processor -- start 2023-10-06T04:33:11.601 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.599+0000 7fb94f4ea700 1 -- start start 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94f4ea700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb948159650 0x7fb9480cb0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94f4ea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9480cb630 0x7fb9480cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94f4ea700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fb9480cef40 0x7fb9480c5250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94f4ea700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9480cf3a0 con 0x7fb948159650 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94f4ea700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9480cc850 con 0x7fb9480cef40 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94f4ea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480cbc40 con 0x7fb9480cb630 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb948159650 0x7fb9480cb0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fb948159650 0x7fb9480cb0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41752/0 (socket says 172.21.15.123:41752) 2023-10-06T04:33:11.604 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 -- 172.21.15.123:0/1592447668 learned_addr learned my addr 172.21.15.123:0/1592447668 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 -- 172.21.15.123:0/1592447668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9480cb630 msgr2=0x7fb9480cea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94ca85700 1 --2- 172.21.15.123:0/1592447668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9480cb630 0x7fb9480cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 --2- 172.21.15.123:0/1592447668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9480cb630 0x7fb9480cea00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94da87700 1 --2- 172.21.15.123:0/1592447668 >> v2:172.21.15.177:3300/0 conn(0x7fb9480cef40 0x7fb9480c5250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 -- 172.21.15.123:0/1592447668 >> v2:172.21.15.177:3300/0 conn(0x7fb9480cef40 msgr2=0x7fb9480c5250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 --2- 172.21.15.123:0/1592447668 >> v2:172.21.15.177:3300/0 conn(0x7fb9480cef40 0x7fb9480c5250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 -- 172.21.15.123:0/1592447668 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9480cbf20 con 0x7fb948159650 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94ca85700 1 --2- 172.21.15.123:0/1592447668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9480cb630 0x7fb9480cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94da87700 1 --2- 172.21.15.123:0/1592447668 >> v2:172.21.15.177:3300/0 conn(0x7fb9480cef40 0x7fb9480c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb94d286700 1 --2- 172.21.15.123:0/1592447668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb948159650 0x7fb9480cb0f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb94806fdb0 tx=0x7fb944009220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7e05588d2a152f0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb93ffff700 1 -- 172.21.15.123:0/1592447668 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944005c10 con 0x7fb948159650 2023-10-06T04:33:11.605 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.600+0000 7fb93ffff700 1 -- 172.21.15.123:0/1592447668 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944005d70 con 0x7fb948159650 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb93ffff700 1 -- 172.21.15.123:0/1592447668 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94400b880 con 0x7fb948159650 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 -- 172.21.15.123:0/1592447668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb948159650 msgr2=0x7fb9480cb0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/1592447668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb948159650 0x7fb9480cb0f0 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7fb94806fdb0 tx=0x7fb944009220 comp rx=0 tx=0).stop 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 -- 172.21.15.123:0/1592447668 shutdown_connections 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/1592447668 >> v2:172.21.15.177:3300/0 conn(0x7fb9480cef40 0x7fb9480c5250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/1592447668 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb948159650 0x7fb9480cb0f0 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/1592447668 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb9480cb630 0x7fb9480cea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 -- 172.21.15.123:0/1592447668 >> 172.21.15.123:0/1592447668 conn(0x7fb948099610 msgr2=0x7fb948063a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 -- 172.21.15.123:0/1592447668 shutdown_connections 2023-10-06T04:33:11.606 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.601+0000 7fb94f4ea700 1 -- 172.21.15.123:0/1592447668 wait complete. 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.602+0000 7fb94f4ea700 1 Processor -- start 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.602+0000 7fb94f4ea700 1 -- start start 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94f4ea700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9480cb630 0x7fb94814e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94f4ea700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9480cef40 0x7fb94815e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94f4ea700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb948159650 0x7fb94815d6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94f4ea700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb94814e790 con 0x7fb9480cb630 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94f4ea700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480c8fb0 con 0x7fb9480cef40 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94f4ea700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9480c83a0 con 0x7fb948159650 2023-10-06T04:33:11.609 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9480cb630 0x7fb94814e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9480cb630 0x7fb94814e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41766/0 (socket says 172.21.15.123:41766) 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 -- 172.21.15.123:0/2307343556 learned_addr learned my addr 172.21.15.123:0/2307343556 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94da87700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb948159650 0x7fb94815d6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94ca85700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9480cef40 0x7fb94815e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 -- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb948159650 msgr2=0x7fb94815d6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb948159650 0x7fb94815d6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 -- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9480cef40 msgr2=0x7fb94815e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9480cef40 0x7fb94815e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94d286700 1 -- 172.21.15.123:0/2307343556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb944009480 con 0x7fb9480cb630 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.603+0000 7fb94ca85700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9480cef40 0x7fb94815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.604+0000 7fb94d286700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9480cb630 0x7fb94814e250 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb944009fd0 tx=0x7fb944016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:11.610 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.604+0000 7fb93d7fa700 1 -- 172.21.15.123:0/2307343556 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944011910 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.604+0000 7fb93d7fa700 1 -- 172.21.15.123:0/2307343556 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb944020070 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.604+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9480c8680 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.604+0000 7fb93d7fa700 1 -- 172.21.15.123:0/2307343556 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb944011ca0 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.604+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9480c8c40 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.605+0000 7fb93d7fa700 1 -- 172.21.15.123:0/2307343556 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fb944022030 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.605+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9480cc200 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.605+0000 7fb93d7fa700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb92c066f30 0x7fb92c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.606+0000 7fb93d7fa700 1 -- 172.21.15.123:0/2307343556 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fb9440860b0 con 0x7fb9480cb630 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.606+0000 7fb94ca85700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb92c066f30 0x7fb92c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.611 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.606+0000 7fb94ca85700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb92c066f30 0x7fb92c0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb938006fd0 tx=0x7fb938008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:11.612 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.610+0000 7fb93d7fa700 1 -- 172.21.15.123:0/2307343556 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fb944053460 con 0x7fb9480cb630 2023-10-06T04:33:11.794 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.792+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config rm", "who": "mds", "name": "debug_mds"} v 0) v1 -- 0x7fb94806f180 con 0x7fb9480cb630 2023-10-06T04:33:11.797 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.795+0000 7fb93d7fa700 1 -- 172.21.15.123:0/2307343556 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mds", "name": "debug_mds"}]=0 v12) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb944056a80 con 0x7fb9480cb630 2023-10-06T04:33:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.797+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb92c066f30 msgr2=0x7fb92c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.797+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb92c066f30 0x7fb92c0693e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fb938006fd0 tx=0x7fb938008040 comp rx=0 tx=0).stop 2023-10-06T04:33:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.797+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9480cb630 msgr2=0x7fb94814e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.797+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9480cb630 0x7fb94814e250 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7fb944009fd0 tx=0x7fb944016000 comp rx=0 tx=0).stop 2023-10-06T04:33:11.800 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 shutdown_connections 2023-10-06T04:33:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fb92c066f30 0x7fb92c0693e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fb9480cb630 0x7fb94814e250 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fb948159650 0x7fb94815d6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 --2- 172.21.15.123:0/2307343556 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fb9480cef40 0x7fb94815e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 >> 172.21.15.123:0/2307343556 conn(0x7fb948099610 msgr2=0x7fb9480c5d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 shutdown_connections 2023-10-06T04:33:11.801 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.798+0000 7fb94f4ea700 1 -- 172.21.15.123:0/2307343556 wait complete. 2023-10-06T04:33:11.812 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-10-06T04:33:11.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.928+0000 7f5cb7f42700 1 Processor -- start 2023-10-06T04:33:11.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.928+0000 7f5cb7f42700 1 -- start start 2023-10-06T04:33:11.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb7f42700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f5cb00c8710 0x7f5cb00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb7f42700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 0x7f5cb0068b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb7f42700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cb0069aa0 con 0x7f5cb00c8340 2023-10-06T04:33:11.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb7f42700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cb00aa500 con 0x7f5cb00c8710 2023-10-06T04:33:11.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb7f42700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0069050 con 0x7f5cb0068670 2023-10-06T04:33:11.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb5cde700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f5cb00c8340 0x7f5cb00a9df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55376/0 (socket says 172.21.15.123:55376) 2023-10-06T04:33:11.934 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb5cde700 1 -- 172.21.15.123:0/345089864 learned_addr learned my addr 172.21.15.123:0/345089864 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3438494052 0 0) 0x7f5cb0069aa0 con 0x7f5cb00c8340 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb64df700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 0x7f5cb0068b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5c9c003570 con 0x7f5cb00c8340 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb54dd700 1 --2- 172.21.15.123:0/345089864 >> v2:172.21.15.177:3300/0 conn(0x7f5cb00c8710 0x7f5cb00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 236185979 0 0) 0x7f5c9c003570 con 0x7f5cb00c8340 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 msgr2=0x7f5cb0068b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 0x7f5cb0068b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 >> v2:172.21.15.177:3300/0 conn(0x7f5cb00c8710 msgr2=0x7f5cb00c8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 --2- 172.21.15.123:0/345089864 >> v2:172.21.15.177:3300/0 conn(0x7f5cb00c8710 0x7f5cb00c8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb006a340 con 0x7f5cb00c8340 2023-10-06T04:33:11.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f5cac001470 con 0x7f5cb00c8340 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 >> v1:172.21.15.123:6789/0 conn(0x7f5cb00c8340 legacy=0x7f5cb00a9df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c9c003a60 0x7f5c9c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.929+0000 7f5cb4cdc700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 0x7f5c9c004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb4cdc700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c004d50 0x7f5c9c0051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb0069050 con 0x7f5c9c004d50 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c00ba60 con 0x7f5c9c003a60 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c9c00bba0 con 0x7f5cb0068670 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb64df700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c004d50 0x7f5c9c0051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb54dd700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 0x7f5c9c004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb5cde700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c9c003a60 0x7f5c9c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb64df700 1 -- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 msgr2=0x7f5c9c004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:11.936 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb64df700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0068670 0x7f5c9c004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb64df700 1 -- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c9c003a60 msgr2=0x7f5c9c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb64df700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c9c003a60 0x7f5c9c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb64df700 1 -- 172.21.15.123:0/345089864 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c9c003570 con 0x7f5c9c004d50 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb5cde700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5c9c003a60 0x7f5c9c003e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb64df700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c004d50 0x7f5c9c0051c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800e210 tx=0x7f5ca800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5fb2cc3da8342caf server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca800ea60 con 0x7f5c9c004d50 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca800f040 con 0x7f5c9c004d50 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.930+0000 7f5cb4cdc700 1 -- 172.21.15.123:0/345089864 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca800ee10 con 0x7f5c9c004d50 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.931+0000 7f5cb7f42700 1 -- 172.21.15.123:0/345089864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c004d50 msgr2=0x7f5c9c0051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.931+0000 7f5cb7f42700 1 --2- 172.21.15.123:0/345089864 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5c9c004d50 0x7f5c9c0051c0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f5ca800e210 tx=0x7f5ca800b040 comp rx=0 tx=0).stop 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.931+0000 7f5cb64df700 1 -- 172.21.15.123:0/345089864 reap_dead start 2023-10-06T04:33:11.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.931+0000 7f5cb7f42700 1 -- 172.21.15.123:0/345089864 shutdown_connections 2023-10-06T04:33:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.931+0000 7f5cb7f42700 1 -- 172.21.15.123:0/345089864 >> 172.21.15.123:0/345089864 conn(0x7f5cb00c9ca0 msgr2=0x7f5cb006da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.931+0000 7f5cb7f42700 1 -- 172.21.15.123:0/345089864 shutdown_connections 2023-10-06T04:33:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.931+0000 7f5cb7f42700 1 -- 172.21.15.123:0/345089864 wait complete. 2023-10-06T04:33:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.932+0000 7f5cb7f42700 1 Processor -- start 2023-10-06T04:33:11.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.932+0000 7f5cb7f42700 1 -- start start 2023-10-06T04:33:11.940 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.932+0000 7f5cb7f42700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0169f70 0x7f5cb016a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb7f42700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb00c37b0 0x7f5cb00c4100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb7f42700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cb00c3c20 0x7f5cb00adf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb7f42700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb00adac0 con 0x7f5cb00c37b0 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb7f42700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb016a880 con 0x7f5cb00c3c20 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb7f42700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cb00aceb0 con 0x7f5cb0169f70 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb00c37b0 0x7f5cb00c4100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb00c37b0 0x7f5cb00c4100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41778/0 (socket says 172.21.15.123:41778) 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 -- 172.21.15.123:0/2353499458 learned_addr learned my addr 172.21.15.123:0/2353499458 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb5cde700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0169f70 0x7f5cb016a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 -- 172.21.15.123:0/2353499458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0169f70 msgr2=0x7f5cb016a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.941 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0169f70 0x7f5cb016a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 -- 172.21.15.123:0/2353499458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cb00c3c20 msgr2=0x7f5cb00adf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cb00c3c20 0x7f5cb00adf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 -- 172.21.15.123:0/2353499458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cb00ad190 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb54dd700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb00c37b0 0x7f5cb00c4100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0006ac0 tx=0x7f5ca00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5ca5ffb700 1 -- 172.21.15.123:0/2353499458 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca000fa10 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5ca5ffb700 1 -- 172.21.15.123:0/2353499458 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ca000fb70 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5ca5ffb700 1 -- 172.21.15.123:0/2353499458 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ca0014440 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cb00ad470 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.933+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cb00c4640 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.934+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cb00ad910 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.935+0000 7f5ca5ffb700 1 -- 172.21.15.123:0/2353499458 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5ca0014830 con 0x7f5cb00c37b0 2023-10-06T04:33:11.942 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.935+0000 7f5ca5ffb700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5c94066f60 0x7f5c94069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:11.943 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.935+0000 7f5ca5ffb700 1 -- 172.21.15.123:0/2353499458 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f5ca0018030 con 0x7f5cb00c37b0 2023-10-06T04:33:11.943 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.935+0000 7f5cb5cde700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5c94066f60 0x7f5c94069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:11.943 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.936+0000 7f5cb5cde700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5c94066f60 0x7f5c94069410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004890 tx=0x7f5cb006a2b0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:11.943 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:11.939+0000 7f5ca5ffb700 1 -- 172.21.15.123:0/2353499458 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f5ca00521a0 con 0x7f5cb00c37b0 2023-10-06T04:33:12.123 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.121+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f5cb00c48c0 con 0x7f5cb00c37b0 2023-10-06T04:33:12.127 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.124+0000 7f5ca5ffb700 1 -- 172.21.15.123:0/2353499458 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v13) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5ca00557c0 con 0x7f5cb00c37b0 2023-10-06T04:33:12.129 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5c94066f60 msgr2=0x7f5c94069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5c94066f60 0x7f5c94069410 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f5cac004890 tx=0x7f5cb006a2b0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb00c37b0 msgr2=0x7f5cb00c4100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb00c37b0 0x7f5cb00c4100 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f5ca0006ac0 tx=0x7f5ca00091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 shutdown_connections 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5c94066f60 0x7f5c94069410 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f5cb00c3c20 0x7f5cb00adf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f5cb0169f70 0x7f5cb016a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 --2- 172.21.15.123:0/2353499458 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f5cb00c37b0 0x7f5cb00c4100 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 >> 172.21.15.123:0/2353499458 conn(0x7f5cb00c9ca0 msgr2=0x7f5cb0158a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:12.130 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 shutdown_connections 2023-10-06T04:33:12.131 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.127+0000 7f5cb7f42700 1 -- 172.21.15.123:0/2353499458 wait complete. 2023-10-06T04:33:12.141 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple' 2023-10-06T04:33:12.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.255+0000 7fc09bfec700 1 Processor -- start 2023-10-06T04:33:12.258 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.256+0000 7fc09bfec700 1 -- start start 2023-10-06T04:33:12.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.256+0000 7fc09bfec700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc09406f080 0x7fc0940c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.256+0000 7fc09bfec700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940c8580 con 0x7fc0941560c0 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.256+0000 7fc09bfec700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0940c8860 con 0x7fc09406e960 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.256+0000 7fc09bfec700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0940c8b40 con 0x7fc09406f080 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc09a589700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fc09406f080 0x7fc0940c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc099d88700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc0941560c0 0x7fc09406e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55382/0 (socket says 172.21.15.123:55382) 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc099d88700 1 -- 172.21.15.123:0/2710379645 learned_addr learned my addr 172.21.15.123:0/2710379645 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc09a589700 1 -- 172.21.15.123:0/2710379645 >> v1:172.21.15.177:6789/0 conn(0x7fc09406e960 legacy=0x7fc0940ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:12.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc09a589700 1 -- 172.21.15.123:0/2710379645 >> v1:172.21.15.123:6789/0 conn(0x7fc0941560c0 legacy=0x7fc09406e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc09a589700 1 -- 172.21.15.123:0/2710379645 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0940c8e20 con 0x7fc09406f080 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc098d86700 1 -- 172.21.15.123:0/2710379645 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816577257 0 0) 0x7fc0940c8580 con 0x7fc0941560c0 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc09a589700 1 --2- 172.21.15.123:0/2710379645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406f080 0x7fc0940c7fa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009e60 tx=0x7fc08c009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1e6c38de2d3264 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc098d86700 1 -- 172.21.15.123:0/2710379645 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c00e070 con 0x7fc09406f080 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc098d86700 1 -- 172.21.15.123:0/2710379645 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc08c00bbf0 con 0x7fc09406f080 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.257+0000 7fc098d86700 1 -- 172.21.15.123:0/2710379645 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc08c012430 con 0x7fc09406f080 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.258+0000 7fc09bfec700 1 -- 172.21.15.123:0/2710379645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406f080 msgr2=0x7fc0940c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.258+0000 7fc09bfec700 1 --2- 172.21.15.123:0/2710379645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406f080 0x7fc0940c7fa0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7fc08c009e60 tx=0x7fc08c009780 comp rx=0 tx=0).stop 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.258+0000 7fc09bfec700 1 -- 172.21.15.123:0/2710379645 shutdown_connections 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.258+0000 7fc09bfec700 1 --2- 172.21.15.123:0/2710379645 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406f080 0x7fc0940c7fa0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.258+0000 7fc09bfec700 1 -- 172.21.15.123:0/2710379645 >> 172.21.15.123:0/2710379645 conn(0x7fc0940cda80 msgr2=0x7fc0940c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:12.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.258+0000 7fc09bfec700 1 -- 172.21.15.123:0/2710379645 shutdown_connections 2023-10-06T04:33:12.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.258+0000 7fc09bfec700 1 -- 172.21.15.123:0/2710379645 wait complete. 2023-10-06T04:33:12.265 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.259+0000 7fc09bfec700 1 Processor -- start 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.259+0000 7fc09bfec700 1 -- start start 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09bfec700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406e960 0x7fc0941543a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09bfec700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc09406f080 0x7fc0940c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09bfec700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0941560c0 0x7fc09414e040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09bfec700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940c6370 con 0x7fc09406e960 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09bfec700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc09414e580 con 0x7fc09406f080 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09bfec700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0940c6560 con 0x7fc0941560c0 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406e960 0x7fc0941543a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:12.266 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406e960 0x7fc0941543a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41794/0 (socket says 172.21.15.123:41794) 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 -- 172.21.15.123:0/805374143 learned_addr learned my addr 172.21.15.123:0/805374143 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09a589700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0941560c0 0x7fc09414e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 -- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0941560c0 msgr2=0x7fc09414e040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0941560c0 0x7fc09414e040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099587700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc09406f080 0x7fc0940c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 -- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc09406f080 msgr2=0x7fc0940c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc09406f080 0x7fc0940c5e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 -- 172.21.15.123:0/805374143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc08c009a20 con 0x7fc09406e960 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09a589700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0941560c0 0x7fc09414e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099587700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc09406f080 0x7fc0940c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:12.267 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc099d88700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406e960 0x7fc0941543a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc08400c5b0 tx=0x7fc084001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc089ffb700 1 -- 172.21.15.123:0/805374143 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084005b30 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc089ffb700 1 -- 172.21.15.123:0/805374143 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc084005c90 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0940c6840 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.260+0000 7fc089ffb700 1 -- 172.21.15.123:0/805374143 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc084007300 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.261+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0940b0120 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.261+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0940cde80 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.262+0000 7fc089ffb700 1 -- 172.21.15.123:0/805374143 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc08401b3b0 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.262+0000 7fc089ffb700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc078066f30 0x7fc0780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.263+0000 7fc089ffb700 1 -- 172.21.15.123:0/805374143 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fc084083970 con 0x7fc09406e960 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.263+0000 7fc099587700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc078066f30 0x7fc0780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:12.268 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.263+0000 7fc099587700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc078066f30 0x7fc0780693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc094159b40 tx=0x7fc090009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:12.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.267+0000 7fc089ffb700 1 -- 172.21.15.123:0/805374143 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc084050d20 con 0x7fc09406e960 2023-10-06T04:33:12.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:12.524+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]} v 0) v1 -- 0x7fc0940c34d0 con 0x7fc09406e960 2023-10-06T04:33:13.459 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.456+0000 7fc089ffb700 1 -- 172.21.15.123:0/805374143 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_recovery_pool.TestRecoveryPool.test_rebuild_simple"]}]=0 v114) v1 ==== 147+0+0 (secure 0 0 0) 0x7fc084017020 con 0x7fc09406e960 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc078066f30 msgr2=0x7fc0780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc078066f30 0x7fc0780693e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fc094159b40 tx=0x7fc090009040 comp rx=0 tx=0).stop 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406e960 msgr2=0x7fc0941543a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406e960 0x7fc0941543a0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fc08400c5b0 tx=0x7fc084001070 comp rx=0 tx=0).stop 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 shutdown_connections 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc078066f30 0x7fc0780693e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc09406e960 0x7fc0941543a0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:13.462 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc0941560c0 0x7fc09414e040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:13.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 --2- 172.21.15.123:0/805374143 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc09406f080 0x7fc0940c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:13.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 >> 172.21.15.123:0/805374143 conn(0x7fc0940cda80 msgr2=0x7fc0940ae2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:13.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 shutdown_connections 2023-10-06T04:33:13.463 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:13.459+0000 7fc09bfec700 1 -- 172.21.15.123:0/805374143 wait complete. 2023-10-06T04:33:13.477 INFO:tasks.cephfs_test_runner:test_rebuild_simple (tasks.cephfs.test_recovery_pool.TestRecoveryPool) ... ok 2023-10-06T04:33:13.477 INFO:tasks.cephfs_test_runner: 2023-10-06T04:33:13.477 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-10-06T04:33:13.477 INFO:tasks.cephfs_test_runner:Ran 1 test in 66.063s 2023-10-06T04:33:13.477 INFO:tasks.cephfs_test_runner: 2023-10-06T04:33:13.477 INFO:tasks.cephfs_test_runner:OK 2023-10-06T04:33:13.478 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-10-06T04:33:13.492 DEBUG:teuthology.run_tasks:Unwinding manager ceph-fuse 2023-10-06T04:33:13.540 INFO:tasks.ceph_fuse:Unmounting ceph-fuse clients... 2023-10-06T04:33:13.540 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.540 DEBUG:teuthology.orchestra.run.smithi123:> ip netns list 2023-10-06T04:33:13.563 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-10-06T04:33:13.564 INFO:teuthology.orchestra.run.smithi123.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-10-06T04:33:13.564 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.564 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-10-06T04:33:13.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.653 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-10-06T04:33:13.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.719 DEBUG:teuthology.orchestra.run.smithi123:> sudo ip link delete ceph-brx 2023-10-06T04:33:13.802 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.802 DEBUG:teuthology.orchestra.run.smithi177:> ip netns list 2023-10-06T04:33:13.824 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-10-06T04:33:13.825 INFO:teuthology.orchestra.run.smithi177.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-10-06T04:33:13.825 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.825 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-10-06T04:33:13.911 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.911 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-10-06T04:33:13.970 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-10-06T04:33:13.970 DEBUG:teuthology.orchestra.run.smithi177:> sudo ip link delete ceph-brx 2023-10-06T04:33:14.058 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-10-06T04:33:14.067 INFO:tasks.ceph.ceph_manager.ceph:Canceling any pending splits or merges... 2023-10-06T04:33:14.067 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:33:14.195 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.193+0000 7f0e655f0700 1 Processor -- start 2023-10-06T04:33:14.196 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.193+0000 7f0e655f0700 1 -- start start 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e655f0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e6006e7d0 0x7f0e60156410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e655f0700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e600c8370 con 0x7f0e600ad880 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e655f0700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e600c7760 con 0x7f0e60156040 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e655f0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e600c7a40 con 0x7f0e6006e7d0 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e6006e7d0 0x7f0e60156410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5f7fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0e6006e7d0 0x7f0e60156410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41802/0 (socket says 172.21.15.123:41802) 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/1774963267 learned_addr learned my addr 172.21.15.123:0/1774963267 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5e7fc700 1 --1- 172.21.15.123:0/1774963267 >> v1:172.21.15.123:6789/0 conn(0x7f0e600ad880 0x7f0e6006d0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55390/0 (socket says 172.21.15.123:55390) 2023-10-06T04:33:14.198 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/1774963267 >> v1:172.21.15.177:6789/0 conn(0x7f0e60156040 legacy=0x7f0e600c6f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/1774963267 >> v1:172.21.15.123:6789/0 conn(0x7f0e600ad880 legacy=0x7f0e6006d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/1774963267 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e600c7d20 con 0x7f0e6006e7d0 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.194+0000 7f0e5f7fe700 1 --2- 172.21.15.123:0/1774963267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e6006e7d0 0x7f0e60156410 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009f20 tx=0x7f0e58009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1673b1d1b4f8ac server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e5dffb700 1 -- 172.21.15.123:0/1774963267 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e580092e0 con 0x7f0e6006e7d0 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e5dffb700 1 -- 172.21.15.123:0/1774963267 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e5800ba40 con 0x7f0e6006e7d0 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e5dffb700 1 -- 172.21.15.123:0/1774963267 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e5800be00 con 0x7f0e6006e7d0 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e655f0700 1 -- 172.21.15.123:0/1774963267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e6006e7d0 msgr2=0x7f0e60156410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e655f0700 1 --2- 172.21.15.123:0/1774963267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e6006e7d0 0x7f0e60156410 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f0e58009f20 tx=0x7f0e58009840 comp rx=0 tx=0).stop 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e655f0700 1 -- 172.21.15.123:0/1774963267 shutdown_connections 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e655f0700 1 --2- 172.21.15.123:0/1774963267 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e6006e7d0 0x7f0e60156410 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e655f0700 1 -- 172.21.15.123:0/1774963267 >> 172.21.15.123:0/1774963267 conn(0x7f0e600680f0 msgr2=0x7f0e600cbf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:14.199 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e655f0700 1 -- 172.21.15.123:0/1774963267 shutdown_connections 2023-10-06T04:33:14.200 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.195+0000 7f0e655f0700 1 -- 172.21.15.123:0/1774963267 wait complete. 2023-10-06T04:33:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.196+0000 7f0e655f0700 1 Processor -- start 2023-10-06T04:33:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e655f0700 1 -- start start 2023-10-06T04:33:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e655f0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e6006e7d0 0x7f0e600c1e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e655f0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e600ad880 0x7f0e600b1730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.202 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e655f0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e60156040 0x7f0e60158b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e655f0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600b12a0 con 0x7f0e60156040 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e655f0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600c23b0 con 0x7f0e600ad880 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e655f0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e600c72f0 con 0x7f0e6006e7d0 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e5f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e60156040 0x7f0e60158b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e5f7fe700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e60156040 0x7f0e60158b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41810/0 (socket says 172.21.15.123:41810) 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/2006233445 learned_addr learned my addr 172.21.15.123:0/2006233445 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e5e7fc700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e600ad880 0x7f0e600b1730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.197+0000 7f0e5effd700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e6006e7d0 0x7f0e600c1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e6006e7d0 msgr2=0x7f0e600c1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e5f7fe700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e6006e7d0 0x7f0e600c1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e600ad880 msgr2=0x7f0e600b1730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.203 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e5f7fe700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e600ad880 0x7f0e600b1730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e5f7fe700 1 -- 172.21.15.123:0/2006233445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e58009ae0 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e5f7fe700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e60156040 0x7f0e60158b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0e6006fb00 tx=0x7f0e58019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e46ffd700 1 -- 172.21.15.123:0/2006233445 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58020070 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e46ffd700 1 -- 172.21.15.123:0/2006233445 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e580092e0 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e600b0970 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e46ffd700 1 -- 172.21.15.123:0/2006233445 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e58007840 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.198+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e600b0f30 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.199+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e600b1450 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.199+0000 7f0e46ffd700 1 -- 172.21.15.123:0/2006233445 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0e58007bf0 con 0x7f0e60156040 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.200+0000 7f0e46ffd700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e4c066f30 0x7f0e4c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.204 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.200+0000 7f0e46ffd700 1 -- 172.21.15.123:0/2006233445 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f0e58087ea0 con 0x7f0e60156040 2023-10-06T04:33:14.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.200+0000 7f0e5effd700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e4c066f30 0x7f0e4c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.205 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.200+0000 7f0e5effd700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e4c066f30 0x7f0e4c0693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009af0 tx=0x7f0e54009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.206 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.204+0000 7f0e46ffd700 1 -- 172.21.15.123:0/2006233445 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0e600b1450 con 0x7f0e60156040 2023-10-06T04:33:14.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.388+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0e600b1450 con 0x7f0e60156040 2023-10-06T04:33:14.391 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.389+0000 7f0e46ffd700 1 -- 172.21.15.123:0/2006233445 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14055 (secure 0 0 0) 0x7f0e600b1450 con 0x7f0e60156040 2023-10-06T04:33:14.392 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:14.392 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":46,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:33:11.444215+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6834/571369576":"2023-10-07T04:33:08.437166+0000","172.21.15.123:6836/571369576":"2023-10-07T04:33:08.437166+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:33:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e4c066f30 msgr2=0x7f0e4c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e4c066f30 0x7f0e4c0693e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f0e54009af0 tx=0x7f0e54009060 comp rx=0 tx=0).stop 2023-10-06T04:33:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e60156040 msgr2=0x7f0e60158b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e60156040 0x7f0e60158b40 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7f0e6006fb00 tx=0x7f0e58019000 comp rx=0 tx=0).stop 2023-10-06T04:33:14.394 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 shutdown_connections 2023-10-06T04:33:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0e4c066f30 0x7f0e4c0693e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0e60156040 0x7f0e60158b40 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0e6006e7d0 0x7f0e600c1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 --2- 172.21.15.123:0/2006233445 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0e600ad880 0x7f0e600b1730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 >> 172.21.15.123:0/2006233445 conn(0x7f0e600680f0 msgr2=0x7f0e600adf10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 shutdown_connections 2023-10-06T04:33:14.395 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.391+0000 7f0e655f0700 1 -- 172.21.15.123:0/2006233445 wait complete. 2023-10-06T04:33:14.406 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-10-06T04:33:14.406 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:33:14.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.522+0000 7fde34185700 1 Processor -- start 2023-10-06T04:33:14.524 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.522+0000 7fde34185700 1 -- start start 2023-10-06T04:33:14.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde34185700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fde2c06e960 0x7fde2c0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.526 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde34185700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 0x7fde2c0ada60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde34185700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde2c0c8580 con 0x7fde2c1560c0 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde34185700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde2c0c8860 con 0x7fde2c06e960 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde34185700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0c8b40 con 0x7fde2c06f080 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31f21700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fde2c1560c0 0x7fde2c06e250 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55404/0 (socket says 172.21.15.123:55404) 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31f21700 1 -- 172.21.15.123:0/3586846467 learned_addr learned my addr 172.21.15.123:0/3586846467 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31720700 1 --2- 172.21.15.123:0/3586846467 >> v2:172.21.15.123:3300/0 conn(0x7fde2c06e960 0x7fde2c0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde32722700 1 --2- 172.21.15.123:0/3586846467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 0x7fde2c0ada60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31720700 1 -- 172.21.15.123:0/3586846467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 msgr2=0x7fde2c0ada60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31720700 1 --2- 172.21.15.123:0/3586846467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 0x7fde2c0ada60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31720700 1 -- 172.21.15.123:0/3586846467 >> v1:172.21.15.123:6789/0 conn(0x7fde2c1560c0 legacy=0x7fde2c06e250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.527 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde30f1f700 1 -- 172.21.15.123:0/3586846467 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285525843 0 0) 0x7fde2c0c8580 con 0x7fde2c1560c0 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31720700 1 -- 172.21.15.123:0/3586846467 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde2c0c8e20 con 0x7fde2c06e960 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde31720700 1 --2- 172.21.15.123:0/3586846467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0c7fa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0098e0 tx=0x7fde1c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45b95d1202366b52 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde30f1f700 1 -- 172.21.15.123:0/3586846467 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c009e80 con 0x7fde2c06e960 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde30f1f700 1 -- 172.21.15.123:0/3586846467 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde1c00b860 con 0x7fde2c06e960 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.523+0000 7fde30f1f700 1 -- 172.21.15.123:0/3586846467 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde1c00bbe0 con 0x7fde2c06e960 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 -- 172.21.15.123:0/3586846467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 msgr2=0x7fde2c0c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 --2- 172.21.15.123:0/3586846467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0c7fa0 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7fde1c0098e0 tx=0x7fde1c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 -- 172.21.15.123:0/3586846467 shutdown_connections 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 --2- 172.21.15.123:0/3586846467 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 0x7fde2c0ada60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 --2- 172.21.15.123:0/3586846467 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0c7fa0 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 -- 172.21.15.123:0/3586846467 >> 172.21.15.123:0/3586846467 conn(0x7fde2c0cda80 msgr2=0x7fde2c0c9a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 -- 172.21.15.123:0/3586846467 shutdown_connections 2023-10-06T04:33:14.528 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.524+0000 7fde34185700 1 -- 172.21.15.123:0/3586846467 wait complete. 2023-10-06T04:33:14.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.525+0000 7fde34185700 1 Processor -- start 2023-10-06T04:33:14.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.525+0000 7fde34185700 1 -- start start 2023-10-06T04:33:14.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde34185700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0afec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde34185700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 0x7fde2c067660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde34185700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2c1560c0 0x7fde2c067ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde34185700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0c5dd0 con 0x7fde2c06e960 2023-10-06T04:33:14.531 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde34185700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0c51c0 con 0x7fde2c1560c0 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde34185700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde2c0c54a0 con 0x7fde2c06f080 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0afec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0afec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41830/0 (socket says 172.21.15.123:41830) 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 -- 172.21.15.123:0/2933916318 learned_addr learned my addr 172.21.15.123:0/2933916318 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 -- 172.21.15.123:0/2933916318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 msgr2=0x7fde2c067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 0x7fde2c067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 -- 172.21.15.123:0/2933916318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2c1560c0 msgr2=0x7fde2c067ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2c1560c0 0x7fde2c067ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 -- 172.21.15.123:0/2933916318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde1c009420 con 0x7fde2c06e960 2023-10-06T04:33:14.532 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.526+0000 7fde31f21700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0afec0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c5b0 tx=0x7fde28001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.527+0000 7fde21ffb700 1 -- 172.21.15.123:0/2933916318 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28005b30 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.527+0000 7fde21ffb700 1 -- 172.21.15.123:0/2933916318 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde28005c90 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.527+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde2c0c5780 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.527+0000 7fde21ffb700 1 -- 172.21.15.123:0/2933916318 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde28007300 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.527+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde2c0b0400 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.528+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde2c06eec0 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.528+0000 7fde21ffb700 1 -- 172.21.15.123:0/2933916318 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fde2801b3b0 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.528+0000 7fde21ffb700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde10066f30 0x7fde100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.529+0000 7fde21ffb700 1 -- 172.21.15.123:0/2933916318 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fde28083920 con 0x7fde2c06e960 2023-10-06T04:33:14.533 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.529+0000 7fde31720700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde10066f30 0x7fde100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.534 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.529+0000 7fde31720700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde10066f30 0x7fde100693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fde2c06ed60 tx=0x7fde1c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.535 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.533+0000 7fde21ffb700 1 -- 172.21.15.123:0/2933916318 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fde28050cd0 con 0x7fde2c06e960 2023-10-06T04:33:14.712 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.709+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde2c0c5f10 con 0x7fde10066f30 2023-10-06T04:33:14.714 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.712+0000 7fde21ffb700 1 -- 172.21.15.123:0/2933916318 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72770 (secure 0 0 0) 0x7fde1c000cd0 con 0x7fde10066f30 2023-10-06T04:33:14.715 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde10066f30 msgr2=0x7fde100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde10066f30 0x7fde100693e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7fde2c06ed60 tx=0x7fde1c017000 comp rx=0 tx=0).stop 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 msgr2=0x7fde2c0afec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0afec0 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7fde2800c5b0 tx=0x7fde28001070 comp rx=0 tx=0).stop 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 shutdown_connections 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fde10066f30 0x7fde100693e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fde2c06e960 0x7fde2c0afec0 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fde2c1560c0 0x7fde2c067ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 --2- 172.21.15.123:0/2933916318 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fde2c06f080 0x7fde2c067660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.717 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 >> 172.21.15.123:0/2933916318 conn(0x7fde2c0cda80 msgr2=0x7fde2c0ad960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:14.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 shutdown_connections 2023-10-06T04:33:14.718 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.714+0000 7fde34185700 1 -- 172.21.15.123:0/2933916318 wait complete. 2023-10-06T04:33:14.718 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:33:14.728 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":95,"stamp":"2023-10-06T04:33:12.909842+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395840,"kb_used_data":17536,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752578880,"statfs":{"total":773094113280,"available":770640773120,"internally_reserved":0,"allocated":17956864,"data_stored":15598722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6321524,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-50,"num_read_kb":-37,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001314"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:04.314498+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:33:04.314498+0000","last_peered":"2023-10-06T04:33:04.314498+0000","last_clean":"2023-10-06T04:33:04.314498+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:33:04.314498+0000","last_undegraded":"2023-10-06T04:33:04.314498+0000","last_fullsized":"2023-10-06T04:33:04.314498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:05.064728+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:33:05.064728+0000","last_peered":"2023-10-06T04:33:05.064728+0000","last_clean":"2023-10-06T04:33:05.064728+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:33:05.064728+0000","last_undegraded":"2023-10-06T04:33:05.064728+0000","last_fullsized":"2023-10-06T04:33:05.064728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.440429+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:33:08.440429+0000","last_peered":"2023-10-06T04:33:08.440429+0000","last_clean":"2023-10-06T04:33:08.440429+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:33:08.440429+0000","last_undegraded":"2023-10-06T04:33:08.440429+0000","last_fullsized":"2023-10-06T04:33:08.440429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:05.064755+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:33:05.064755+0000","last_peered":"2023-10-06T04:33:05.064755+0000","last_clean":"2023-10-06T04:33:05.064755+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:33:05.064755+0000","last_undegraded":"2023-10-06T04:33:05.064755+0000","last_fullsized":"2023-10-06T04:33:05.064755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442473+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:33:08.442473+0000","last_peered":"2023-10-06T04:33:08.442473+0000","last_clean":"2023-10-06T04:33:08.442473+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:33:08.442473+0000","last_undegraded":"2023-10-06T04:33:08.442473+0000","last_fullsized":"2023-10-06T04:33:08.442473+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442383+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:33:08.442383+0000","last_peered":"2023-10-06T04:33:08.442383+0000","last_clean":"2023-10-06T04:33:08.442383+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:33:08.442383+0000","last_undegraded":"2023-10-06T04:33:08.442383+0000","last_fullsized":"2023-10-06T04:33:08.442383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.443033+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:33:08.443033+0000","last_peered":"2023-10-06T04:33:08.443033+0000","last_clean":"2023-10-06T04:33:08.443033+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:33:08.443033+0000","last_undegraded":"2023-10-06T04:33:08.443033+0000","last_fullsized":"2023-10-06T04:33:08.443033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:04.314114+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:33:04.314114+0000","last_peered":"2023-10-06T04:33:04.314114+0000","last_clean":"2023-10-06T04:33:04.314114+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:33:04.314114+0000","last_undegraded":"2023-10-06T04:33:04.314114+0000","last_fullsized":"2023-10-06T04:33:04.314114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442063+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:33:08.442063+0000","last_peered":"2023-10-06T04:33:08.442063+0000","last_clean":"2023-10-06T04:33:08.442063+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:33:08.442063+0000","last_undegraded":"2023-10-06T04:33:08.442063+0000","last_fullsized":"2023-10-06T04:33:08.442063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300424,"kb_used_data":3136,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071416,"statfs":{"total":96636764160,"available":96329129984,"internally_reserved":0,"allocated":3211264,"data_stored":2911142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.434},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.504},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297796,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074044,"statfs":{"total":96636764160,"available":96331821056,"internally_reserved":0,"allocated":520192,"data_stored":224122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.502}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297796,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074044,"statfs":{"total":96636764160,"available":96331821056,"internally_reserved":0,"allocated":520192,"data_stored":223658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.255},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.377},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.755}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298376,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073464,"statfs":{"total":96636764160,"available":96331227136,"internally_reserved":0,"allocated":1114112,"data_stored":813922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":252796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.308},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.28000000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.375}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55200000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301924,"kb_used_data":4636,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069916,"statfs":{"total":96636764160,"available":96327593984,"internally_reserved":0,"allocated":4747264,"data_stored":4447122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.317},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.34999999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":503808,"data_stored":217529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.496},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.436}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303920,"kb_used_data":6632,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067920,"statfs":{"total":96636764160,"available":96325550080,"internally_reserved":0,"allocated":6791168,"data_stored":6508431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.749}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:33:14.730 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:33:14.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.844+0000 7fec846ed700 1 Processor -- start 2023-10-06T04:33:14.847 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.845+0000 7fec846ed700 1 -- start start 2023-10-06T04:33:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.845+0000 7fec846ed700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec7c0ab230 0x7fec7c0aec10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.845+0000 7fec846ed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0ace40 0x7fec7c0ad2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.849 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.845+0000 7fec846ed700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec7c0abe30 con 0x7fec7c0aae60 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.845+0000 7fec846ed700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec7c0ac9b0 con 0x7fec7c0ab230 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.845+0000 7fec846ed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c0ac080 con 0x7fec7c0ace40 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec7c0ab230 0x7fec7c0aec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fec7c0ab230 0x7fec7c0aec10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41836/0 (socket says 172.21.15.123:41836) 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 -- 172.21.15.123:0/343388514 learned_addr learned my addr 172.21.15.123:0/343388514 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec82c8a700 1 --2- 172.21.15.123:0/343388514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0ace40 0x7fec7c0ad2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 -- 172.21.15.123:0/343388514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0ace40 msgr2=0x7fec7c0ad2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:14.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 --2- 172.21.15.123:0/343388514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0ace40 0x7fec7c0ad2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 -- 172.21.15.123:0/343388514 >> v1:172.21.15.177:6789/0 conn(0x7fec7c0aae60 legacy=0x7fec7c0afbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 -- 172.21.15.123:0/343388514 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec7c0ac360 con 0x7fec7c0ab230 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81c88700 1 --2- 172.21.15.123:0/343388514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ab230 0x7fec7c0aec10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec780099b0 tx=0x7fec78009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe45052151977723 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81487700 1 -- 172.21.15.123:0/343388514 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800b860 con 0x7fec7c0ab230 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81487700 1 -- 172.21.15.123:0/343388514 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7800b9c0 con 0x7fec7c0ab230 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.846+0000 7fec81487700 1 -- 172.21.15.123:0/343388514 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7800bd60 con 0x7fec7c0ab230 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 -- 172.21.15.123:0/343388514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ab230 msgr2=0x7fec7c0aec10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 --2- 172.21.15.123:0/343388514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ab230 0x7fec7c0aec10 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fec780099b0 tx=0x7fec78009250 comp rx=0 tx=0).stop 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 -- 172.21.15.123:0/343388514 shutdown_connections 2023-10-06T04:33:14.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 --2- 172.21.15.123:0/343388514 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0ace40 0x7fec7c0ad2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 --2- 172.21.15.123:0/343388514 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ab230 0x7fec7c0aec10 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 -- 172.21.15.123:0/343388514 >> 172.21.15.123:0/343388514 conn(0x7fec7c097410 msgr2=0x7fec7c067e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 -- 172.21.15.123:0/343388514 shutdown_connections 2023-10-06T04:33:14.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.847+0000 7fec846ed700 1 -- 172.21.15.123:0/343388514 wait complete. 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.848+0000 7fec846ed700 1 Processor -- start 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.848+0000 7fec846ed700 1 -- start start 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0aae60 0x7fec7c159700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec7c0ab230 0x7fec7c09b090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ace40 0x7fec7c09b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c098190 con 0x7fec7c0ace40 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c09bb10 con 0x7fec7c0ab230 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec7c159fe0 con 0x7fec7c0aae60 2023-10-06T04:33:14.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ace40 0x7fec7c09b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ace40 0x7fec7c09b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41844/0 (socket says 172.21.15.123:41844) 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 -- 172.21.15.123:0/2679701037 learned_addr learned my addr 172.21.15.123:0/2679701037 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 -- 172.21.15.123:0/2679701037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0aae60 msgr2=0x7fec7c159700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0aae60 0x7fec7c159700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 -- 172.21.15.123:0/2679701037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec7c0ab230 msgr2=0x7fec7c09b090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec7c0ab230 0x7fec7c09b090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 -- 172.21.15.123:0/2679701037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec780094f0 con 0x7fec7c0ace40 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec82c8a700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ace40 0x7fec7c09b5d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec74007f20 tx=0x7fec74009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec6e7fc700 1 -- 172.21.15.123:0/2679701037 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec7400eca0 con 0x7fec7c0ace40 2023-10-06T04:33:14.856 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec6e7fc700 1 -- 172.21.15.123:0/2679701037 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec7400ee20 con 0x7fec7c0ace40 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec7c15a2c0 con 0x7fec7c0ace40 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec6e7fc700 1 -- 172.21.15.123:0/2679701037 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec74014670 con 0x7fec7c0ace40 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.849+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec7c15a5a0 con 0x7fec7c0ace40 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.850+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec7c06d550 con 0x7fec7c0ace40 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.851+0000 7fec6e7fc700 1 -- 172.21.15.123:0/2679701037 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fec740149e0 con 0x7fec7c0ace40 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.851+0000 7fec6e7fc700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fec60066f30 0x7fec600693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.851+0000 7fec6e7fc700 1 -- 172.21.15.123:0/2679701037 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fec74084b20 con 0x7fec7c0ace40 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.851+0000 7fec82489700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fec60066f30 0x7fec600693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:14.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.851+0000 7fec82489700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fec60066f30 0x7fec600693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fec70001730 tx=0x7fec70007b40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:14.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:14.855+0000 7fec6e7fc700 1 -- 172.21.15.123:0/2679701037 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fec74051ed0 con 0x7fec7c0ace40 2023-10-06T04:33:15.037 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.035+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fec7c0982d0 con 0x7fec60066f30 2023-10-06T04:33:15.040 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.038+0000 7fec6e7fc700 1 -- 172.21.15.123:0/2679701037 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72770 (secure 0 0 0) 0x7fec7000c430 con 0x7fec60066f30 2023-10-06T04:33:15.041 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fec60066f30 msgr2=0x7fec600693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fec60066f30 0x7fec600693e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7fec70001730 tx=0x7fec70007b40 comp rx=0 tx=0).stop 2023-10-06T04:33:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ace40 msgr2=0x7fec7c09b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ace40 0x7fec7c09b5d0 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fec74007f20 tx=0x7fec74009860 comp rx=0 tx=0).stop 2023-10-06T04:33:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 shutdown_connections 2023-10-06T04:33:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fec60066f30 0x7fec600693e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fec7c0ace40 0x7fec7c09b5d0 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fec7c0aae60 0x7fec7c159700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 --2- 172.21.15.123:0/2679701037 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fec7c0ab230 0x7fec7c09b090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 >> 172.21.15.123:0/2679701037 conn(0x7fec7c097410 msgr2=0x7fec7c0b37b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 shutdown_connections 2023-10-06T04:33:15.044 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.040+0000 7fec846ed700 1 -- 172.21.15.123:0/2679701037 wait complete. 2023-10-06T04:33:15.045 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:33:15.054 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-10-06T04:33:14.910113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395840,"kb_used_data":17536,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752578880,"statfs":{"total":773094113280,"available":770640773120,"internally_reserved":0,"allocated":17956864,"data_stored":15598722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6321524,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-50,"num_read_kb":-37,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.471041"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:04.314498+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:33:04.314498+0000","last_peered":"2023-10-06T04:33:04.314498+0000","last_clean":"2023-10-06T04:33:04.314498+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:33:04.314498+0000","last_undegraded":"2023-10-06T04:33:04.314498+0000","last_fullsized":"2023-10-06T04:33:04.314498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:05.064728+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:33:05.064728+0000","last_peered":"2023-10-06T04:33:05.064728+0000","last_clean":"2023-10-06T04:33:05.064728+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:33:05.064728+0000","last_undegraded":"2023-10-06T04:33:05.064728+0000","last_fullsized":"2023-10-06T04:33:05.064728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.440429+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:33:08.440429+0000","last_peered":"2023-10-06T04:33:08.440429+0000","last_clean":"2023-10-06T04:33:08.440429+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:33:08.440429+0000","last_undegraded":"2023-10-06T04:33:08.440429+0000","last_fullsized":"2023-10-06T04:33:08.440429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:05.064755+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:33:05.064755+0000","last_peered":"2023-10-06T04:33:05.064755+0000","last_clean":"2023-10-06T04:33:05.064755+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:33:05.064755+0000","last_undegraded":"2023-10-06T04:33:05.064755+0000","last_fullsized":"2023-10-06T04:33:05.064755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442473+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:33:08.442473+0000","last_peered":"2023-10-06T04:33:08.442473+0000","last_clean":"2023-10-06T04:33:08.442473+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:33:08.442473+0000","last_undegraded":"2023-10-06T04:33:08.442473+0000","last_fullsized":"2023-10-06T04:33:08.442473+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442383+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:33:08.442383+0000","last_peered":"2023-10-06T04:33:08.442383+0000","last_clean":"2023-10-06T04:33:08.442383+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:33:08.442383+0000","last_undegraded":"2023-10-06T04:33:08.442383+0000","last_fullsized":"2023-10-06T04:33:08.442383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.443033+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:33:08.443033+0000","last_peered":"2023-10-06T04:33:08.443033+0000","last_clean":"2023-10-06T04:33:08.443033+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:33:08.443033+0000","last_undegraded":"2023-10-06T04:33:08.443033+0000","last_fullsized":"2023-10-06T04:33:08.443033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:04.314114+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:33:04.314114+0000","last_peered":"2023-10-06T04:33:04.314114+0000","last_clean":"2023-10-06T04:33:04.314114+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:33:04.314114+0000","last_undegraded":"2023-10-06T04:33:04.314114+0000","last_fullsized":"2023-10-06T04:33:04.314114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442063+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:33:08.442063+0000","last_peered":"2023-10-06T04:33:08.442063+0000","last_clean":"2023-10-06T04:33:08.442063+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:33:08.442063+0000","last_undegraded":"2023-10-06T04:33:08.442063+0000","last_fullsized":"2023-10-06T04:33:08.442063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300424,"kb_used_data":3136,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071416,"statfs":{"total":96636764160,"available":96329129984,"internally_reserved":0,"allocated":3211264,"data_stored":2911142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.434},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.504},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297796,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074044,"statfs":{"total":96636764160,"available":96331821056,"internally_reserved":0,"allocated":520192,"data_stored":224122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.502}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297796,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074044,"statfs":{"total":96636764160,"available":96331821056,"internally_reserved":0,"allocated":520192,"data_stored":223658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.255},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.377},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.755}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298376,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073464,"statfs":{"total":96636764160,"available":96331227136,"internally_reserved":0,"allocated":1114112,"data_stored":813922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":252796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.308},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.28000000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.375}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55200000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301924,"kb_used_data":4636,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069916,"statfs":{"total":96636764160,"available":96327593984,"internally_reserved":0,"allocated":4747264,"data_stored":4447122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.317},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.34999999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":503808,"data_stored":217529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.496},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.436}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303920,"kb_used_data":6632,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067920,"statfs":{"total":96636764160,"available":96325550080,"internally_reserved":0,"allocated":6791168,"data_stored":6508431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.749}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:33:15.056 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-10-06T04:33:15.056 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:33:15.173 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.171+0000 7f26ebfd0700 1 Processor -- start 2023-10-06T04:33:15.174 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.171+0000 7f26ebfd0700 1 -- start start 2023-10-06T04:33:15.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26ebfd0700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26e4155780 0x7f26e4068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.176 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26ebfd0700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f26e4068bb0 0x7f26e40c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26ebfd0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e40c7460 0x7f26e4159460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26ebfd0700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e4069020 con 0x7f26e4155780 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26ebfd0700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e40c7c50 con 0x7f26e4068bb0 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26ebfd0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40c7f30 con 0x7f26e40c7460 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26e9d6c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26e4155780 0x7f26e4068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26e9d6c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f26e4155780 0x7f26e4068670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41848/0 (socket says 172.21.15.123:41848) 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.172+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2033562688 learned_addr learned my addr 172.21.15.123:0/2033562688 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2033562688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e40c7460 msgr2=0x7f26e4159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26ea56d700 1 --2- 172.21.15.123:0/2033562688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e40c7460 0x7f26e4159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.177 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e956b700 1 --2- 172.21.15.123:0/2033562688 >> v2:172.21.15.177:3300/0 conn(0x7f26e4068bb0 0x7f26e40c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e9d6c700 1 --2- 172.21.15.123:0/2033562688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e40c7460 0x7f26e4159460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2033562688 >> v2:172.21.15.177:3300/0 conn(0x7f26e4068bb0 msgr2=0x7f26e40c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e9d6c700 1 --2- 172.21.15.123:0/2033562688 >> v2:172.21.15.177:3300/0 conn(0x7f26e4068bb0 0x7f26e40c6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2033562688 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40c8210 con 0x7f26e4155780 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e9d6c700 1 --2- 172.21.15.123:0/2033562688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4155780 0x7f26e4068670 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26e406ea70 tx=0x7f26d4009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf26ed456ce8d23 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e8d6a700 1 -- 172.21.15.123:0/2033562688 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d400b860 con 0x7f26e4155780 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e8d6a700 1 -- 172.21.15.123:0/2033562688 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d400b9c0 con 0x7f26e4155780 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26e8d6a700 1 -- 172.21.15.123:0/2033562688 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d400bd60 con 0x7f26e4155780 2023-10-06T04:33:15.178 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2033562688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4155780 msgr2=0x7f26e4068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.173+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2033562688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4155780 0x7f26e4068670 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f26e406ea70 tx=0x7f26d4009250 comp rx=0 tx=0).stop 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.174+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2033562688 shutdown_connections 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.174+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2033562688 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e40c7460 0x7f26e4159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.174+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2033562688 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4155780 0x7f26e4068670 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.174+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2033562688 >> v2:172.21.15.177:3300/0 conn(0x7f26e4068bb0 0x7f26e40c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.174+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2033562688 >> 172.21.15.123:0/2033562688 conn(0x7f26e40c9c90 msgr2=0x7f26e406da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.174+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2033562688 shutdown_connections 2023-10-06T04:33:15.179 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.174+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2033562688 wait complete. 2023-10-06T04:33:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 Processor -- start 2023-10-06T04:33:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 -- start start 2023-10-06T04:33:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4068bb0 0x7f26e40ab110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e40c7460 0x7f26e40afc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e4155780 0x7f26e4169cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40b1b10 con 0x7f26e4068bb0 2023-10-06T04:33:15.182 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40b1df0 con 0x7f26e40c7460 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.175+0000 7f26ebfd0700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e40b20d0 con 0x7f26e4155780 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4068bb0 0x7f26e40ab110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4068bb0 0x7f26e40ab110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41856/0 (socket says 172.21.15.123:41856) 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2844751251 learned_addr learned my addr 172.21.15.123:0/2844751251 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26ea56d700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e4155780 0x7f26e4169cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e956b700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e40c7460 0x7f26e40afc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e4155780 msgr2=0x7f26e4169cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e4155780 0x7f26e4169cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e40c7460 msgr2=0x7f26e40afc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e40c7460 0x7f26e40afc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.183 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 -- 172.21.15.123:0/2844751251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26d40094f0 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e956b700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e40c7460 0x7f26e40afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26e9d6c700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4068bb0 0x7f26e40ab110 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f26e406d230 tx=0x7f26d4017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26d9ffb700 1 -- 172.21.15.123:0/2844751251 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4011e70 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26d9ffb700 1 -- 172.21.15.123:0/2844751251 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26d4024440 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e40b23b0 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26d9ffb700 1 -- 172.21.15.123:0/2844751251 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26d4024800 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.176+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e414f320 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.177+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e406cc70 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.178+0000 7f26d9ffb700 1 -- 172.21.15.123:0/2844751251 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f26d401a420 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.178+0000 7f26d9ffb700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f26c8066f30 0x7f26c80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.178+0000 7f26d9ffb700 1 -- 172.21.15.123:0/2844751251 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f26d4086350 con 0x7f26e4068bb0 2023-10-06T04:33:15.184 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.178+0000 7f26e956b700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f26c8066f30 0x7f26c80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.179+0000 7f26e956b700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f26c8066f30 0x7f26c80693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26e40c2f00 tx=0x7f26e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:15.185 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.182+0000 7f26d9ffb700 1 -- 172.21.15.123:0/2844751251 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f26d4053700 con 0x7f26e4068bb0 2023-10-06T04:33:15.361 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.359+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f26e416d2d0 con 0x7f26c8066f30 2023-10-06T04:33:15.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.362+0000 7f26d9ffb700 1 -- 172.21.15.123:0/2844751251 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72770 (secure 0 0 0) 0x7f26e416d2d0 con 0x7f26c8066f30 2023-10-06T04:33:15.365 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f26c8066f30 msgr2=0x7f26c80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f26c8066f30 0x7f26c80693e0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f26e40c2f00 tx=0x7f26e0008040 comp rx=0 tx=0).stop 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4068bb0 msgr2=0x7f26e40ab110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4068bb0 0x7f26e40ab110 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7f26e406d230 tx=0x7f26d4017000 comp rx=0 tx=0).stop 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 shutdown_connections 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f26c8066f30 0x7f26c80693e0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f26e4068bb0 0x7f26e40ab110 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f26e4155780 0x7f26e4169cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 --2- 172.21.15.123:0/2844751251 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f26e40c7460 0x7f26e40afc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 >> 172.21.15.123:0/2844751251 conn(0x7f26e40c9c90 msgr2=0x7f26e4151220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:15.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 shutdown_connections 2023-10-06T04:33:15.368 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.364+0000 7f26ebfd0700 1 -- 172.21.15.123:0/2844751251 wait complete. 2023-10-06T04:33:15.368 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:33:15.378 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":96,"stamp":"2023-10-06T04:33:14.910113+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":210,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":2395840,"kb_used_data":17536,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752578880,"statfs":{"total":773094113280,"available":770640773120,"internally_reserved":0,"allocated":17956864,"data_stored":15598722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":11,"apply_latency_ms":11,"commit_latency_ns":11000000,"apply_latency_ns":11000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6321524,"num_objects":-15,"num_object_clones":0,"num_object_copies":-30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-15,"num_whiteouts":0,"num_read":-50,"num_read_kb":-37,"num_write":-46,"num_write_kb":-6192,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.471041"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:04.314498+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:33:04.314498+0000","last_peered":"2023-10-06T04:33:04.314498+0000","last_clean":"2023-10-06T04:33:04.314498+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:33:04.314498+0000","last_undegraded":"2023-10-06T04:33:04.314498+0000","last_fullsized":"2023-10-06T04:33:04.314498+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:05.064728+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:33:05.064728+0000","last_peered":"2023-10-06T04:33:05.064728+0000","last_clean":"2023-10-06T04:33:05.064728+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:33:05.064728+0000","last_undegraded":"2023-10-06T04:33:05.064728+0000","last_fullsized":"2023-10-06T04:33:05.064728+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":34,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.440429+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:33:08.440429+0000","last_peered":"2023-10-06T04:33:08.440429+0000","last_clean":"2023-10-06T04:33:08.440429+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:33:08.440429+0000","last_undegraded":"2023-10-06T04:33:08.440429+0000","last_fullsized":"2023-10-06T04:33:08.440429+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":30,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:05.064755+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:33:05.064755+0000","last_peered":"2023-10-06T04:33:05.064755+0000","last_clean":"2023-10-06T04:33:05.064755+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:33:05.064755+0000","last_undegraded":"2023-10-06T04:33:05.064755+0000","last_fullsized":"2023-10-06T04:33:05.064755+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":185,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442473+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:33:08.442473+0000","last_peered":"2023-10-06T04:33:08.442473+0000","last_clean":"2023-10-06T04:33:08.442473+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:33:08.442473+0000","last_undegraded":"2023-10-06T04:33:08.442473+0000","last_fullsized":"2023-10-06T04:33:08.442473+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442383+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:33:08.442383+0000","last_peered":"2023-10-06T04:33:08.442383+0000","last_clean":"2023-10-06T04:33:08.442383+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:33:08.442383+0000","last_undegraded":"2023-10-06T04:33:08.442383+0000","last_fullsized":"2023-10-06T04:33:08.442383+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":32,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.443033+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:33:08.443033+0000","last_peered":"2023-10-06T04:33:08.443033+0000","last_clean":"2023-10-06T04:33:08.443033+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:33:08.443033+0000","last_undegraded":"2023-10-06T04:33:08.443033+0000","last_fullsized":"2023-10-06T04:33:08.443033+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":31,"reported_epoch":43,"state":"active+clean","last_fresh":"2023-10-06T04:33:04.314114+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:33:04.314114+0000","last_peered":"2023-10-06T04:33:04.314114+0000","last_clean":"2023-10-06T04:33:04.314114+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:33:04.314114+0000","last_undegraded":"2023-10-06T04:33:04.314114+0000","last_fullsized":"2023-10-06T04:33:04.314114+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":33,"reported_epoch":44,"state":"active+clean","last_fresh":"2023-10-06T04:33:08.442063+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:33:08.442063+0000","last_peered":"2023-10-06T04:33:08.442063+0000","last_clean":"2023-10-06T04:33:08.442063+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:33:08.442063+0000","last_undegraded":"2023-10-06T04:33:08.442063+0000","last_fullsized":"2023-10-06T04:33:08.442063+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":300424,"kb_used_data":3136,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94071416,"statfs":{"total":96636764160,"available":96329129984,"internally_reserved":0,"allocated":3211264,"data_stored":2911142,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.377}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.54000000000000004},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.57499999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.90900000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.46400000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.434},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.48899999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.504},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.67200000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.55100000000000005}]}]},{"osd":6,"up_from":13,"seq":55834574873,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297796,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074044,"statfs":{"total":96636764160,"available":96331821056,"internally_reserved":0,"allocated":520192,"data_stored":224122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.43099999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.502}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.48099999999999998},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.53900000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.52300000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.60699999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.65700000000000003}]}]},{"osd":1,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297796,"kb_used_data":508,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074044,"statfs":{"total":96636764160,"available":96331821056,"internally_reserved":0,"allocated":520192,"data_stored":223658,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.255},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.27900000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.32700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.377},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.40400000000000003}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.69199999999999995}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.755}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.71799999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.64800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834574874,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298376,"kb_used_data":1088,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073464,"statfs":{"total":96636764160,"available":96331227136,"internally_reserved":0,"allocated":1114112,"data_stored":813922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.33300000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.40100000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.34799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.54900000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.75800000000000001}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.629}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.78100000000000003}]}]},{"osd":2,"up_from":13,"seq":55834574874,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297824,"kb_used_data":536,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074016,"statfs":{"total":96636764160,"available":96331792384,"internally_reserved":0,"allocated":548864,"data_stored":252796,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.308},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.28000000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.29499999999999998},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.34799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.32800000000000001},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.375}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.54500000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.58999999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.55200000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.67500000000000004},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.66800000000000004}]}]},{"osd":3,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":301924,"kb_used_data":4636,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94069916,"statfs":{"total":96636764160,"available":96327593984,"internally_reserved":0,"allocated":4747264,"data_stored":4447122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.317},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.46100000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.29899999999999999},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.34999999999999998}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.41999999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.53200000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.54100000000000004}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.55600000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.59699999999999998}]}]},{"osd":4,"up_from":13,"seq":55834574875,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297780,"kb_used_data":492,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074060,"statfs":{"total":96636764160,"available":96331837440,"internally_reserved":0,"allocated":503808,"data_stored":217529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.46800000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.58499999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.50700000000000001}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.496},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.48399999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.436}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.63600000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.54100000000000004}]}]},{"osd":5,"up_from":13,"seq":55834574874,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":303920,"kb_used_data":6632,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94067920,"statfs":{"total":96636764160,"available":96325550080,"internally_reserved":0,"allocated":6791168,"data_stored":6508431,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.749}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.76300000000000001},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.80800000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.78700000000000003},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.71799999999999997}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.70099999999999996}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.58599999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.61199999999999999}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.68200000000000005},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64000000000000001}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:33:15.380 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-10-06T04:33:15.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.498+0000 7fc7c16d7700 1 Processor -- start 2023-10-06T04:33:15.500 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.498+0000 7fc7c16d7700 1 -- start start 2023-10-06T04:33:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.498+0000 7fc7c16d7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 0x7fc7bc0af110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.498+0000 7fc7c16d7700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc7bc0af650 0x7fc7bc15d060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.498+0000 7fc7c16d7700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7bc0ac9c0 con 0x7fc7bc159300 2023-10-06T04:33:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.498+0000 7fc7c16d7700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7bc0abdb0 con 0x7fc7bc0af650 2023-10-06T04:33:15.504 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.498+0000 7fc7c16d7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0ac090 con 0x7fc7bc1596d0 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7baffd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc7bc159300 0x7fc7bc0cca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55412/0 (socket says 172.21.15.123:55412) 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7baffd700 1 -- 172.21.15.123:0/1626736935 learned_addr learned my addr 172.21.15.123:0/1626736935 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1828154437 0 0) 0x7fc7bc0ac9c0 con 0x7fc7bc159300 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7a4003570 con 0x7fc7bc159300 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7ba7fc700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 0x7fc7bc0af110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7bb7fe700 1 --2- 172.21.15.123:0/1626736935 >> v2:172.21.15.177:3300/0 conn(0x7fc7bc0af650 0x7fc7bc15d060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1271243887 0 0) 0x7fc7a4003570 con 0x7fc7bc159300 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 msgr2=0x7fc7bc0af110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 0x7fc7bc0af110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.505 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 >> v2:172.21.15.177:3300/0 conn(0x7fc7bc0af650 msgr2=0x7fc7bc15d060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 --2- 172.21.15.123:0/1626736935 >> v2:172.21.15.177:3300/0 conn(0x7fc7bc0af650 0x7fc7bc15d060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc0ac370 con 0x7fc7bc159300 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fc7ac001470 con 0x7fc7bc159300 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 >> v1:172.21.15.123:6789/0 conn(0x7fc7bc159300 legacy=0x7fc7bc0cca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a4003ae0 0x7fc7a4003ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 0x7fc7a4004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a4004d50 0x7fc7a40051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0abdb0 con 0x7fc7a4003ae0 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc0ac090 con 0x7fc7a4004d50 2023-10-06T04:33:15.506 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a400bb40 con 0x7fc7bc1596d0 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7baffd700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a4003ae0 0x7fc7a4003ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7ba7fc700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 0x7fc7a4004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.499+0000 7fc7bb7fe700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a4004d50 0x7fc7a40051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7baffd700 1 -- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 msgr2=0x7fc7a4004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7baffd700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7bc1596d0 0x7fc7a4004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7baffd700 1 -- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a4004d50 msgr2=0x7fc7a40051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7baffd700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7a4004d50 0x7fc7a40051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7baffd700 1 -- 172.21.15.123:0/1626736935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a4003570 con 0x7fc7a4003ae0 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7baffd700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a4003ae0 0x7fc7a4003ef0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c2a0 tx=0x7fc7bc0ac930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6dd8308b6194af6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:15.507 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac00cb70 con 0x7fc7a4003ae0 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ac00ccf0 con 0x7fc7a4003ae0 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7b9ffb700 1 -- 172.21.15.123:0/1626736935 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ac005b70 con 0x7fc7a4003ae0 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7c16d7700 1 -- 172.21.15.123:0/1626736935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a4003ae0 msgr2=0x7fc7a4003ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7c16d7700 1 --2- 172.21.15.123:0/1626736935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7a4003ae0 0x7fc7a4003ef0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac00c2a0 tx=0x7fc7bc0ac930 comp rx=0 tx=0).stop 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7bb7fe700 1 -- 172.21.15.123:0/1626736935 reap_dead start 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7c16d7700 1 -- 172.21.15.123:0/1626736935 shutdown_connections 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7c16d7700 1 -- 172.21.15.123:0/1626736935 >> 172.21.15.123:0/1626736935 conn(0x7fc7bc06a1c0 msgr2=0x7fc7bc0cc350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7c16d7700 1 -- 172.21.15.123:0/1626736935 shutdown_connections 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.500+0000 7fc7c16d7700 1 -- 172.21.15.123:0/1626736935 wait complete. 2023-10-06T04:33:15.508 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.502+0000 7fc7c16d7700 1 Processor -- start 2023-10-06T04:33:15.509 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.502+0000 7fc7c16d7700 1 -- start start 2023-10-06T04:33:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7c16d7700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a4003ae0 0x7fc7bc149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7c16d7700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7bc15ebe0 0x7fc7bc159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7c16d7700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7bc06e170 0x7fc7bc06e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7c16d7700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc14a490 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7c16d7700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc06eb20 con 0x7fc7bc06e170 2023-10-06T04:33:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7c16d7700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7bc14b590 con 0x7fc7a4003ae0 2023-10-06T04:33:15.511 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7bc15ebe0 0x7fc7bc159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7bc15ebe0 0x7fc7bc159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41884/0 (socket says 172.21.15.123:41884) 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 -- 172.21.15.123:0/2880167721 learned_addr learned my addr 172.21.15.123:0/2880167721 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 -- 172.21.15.123:0/2880167721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a4003ae0 msgr2=0x7fc7bc149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a4003ae0 0x7fc7bc149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 -- 172.21.15.123:0/2880167721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7bc06e170 msgr2=0x7fc7bc06e5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7bc06e170 0x7fc7bc06e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.503+0000 7fc7ba7fc700 1 -- 172.21.15.123:0/2880167721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7bc15a410 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.504+0000 7fc7ba7fc700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7bc15ebe0 0x7fc7bc159b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00119c0 tx=0x7fc7b0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:15.512 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.504+0000 7fc7a2ffd700 1 -- 172.21.15.123:0/2880167721 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000f6d0 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.504+0000 7fc7a2ffd700 1 -- 172.21.15.123:0/2880167721 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7b000f830 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.504+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7bc15a6f0 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.504+0000 7fc7a2ffd700 1 -- 172.21.15.123:0/2880167721 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7b000fc30 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.504+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7bc14b1b0 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.504+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7bc06c860 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.506+0000 7fc7a2ffd700 1 -- 172.21.15.123:0/2880167721 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc7b000fd90 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.506+0000 7fc7a2ffd700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc7a8066f30 0x7fc7a80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.506+0000 7fc7a2ffd700 1 -- 172.21.15.123:0/2880167721 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fc7b0084e80 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.507+0000 7fc7baffd700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc7a8066f30 0x7fc7a80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:15.513 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.507+0000 7fc7baffd700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc7a8066f30 0x7fc7a80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac001ea0 tx=0x7fc7ac015000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:15.514 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.510+0000 7fc7a2ffd700 1 -- 172.21.15.123:0/2880167721 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc7b00522b0 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.693+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc7bc14b6d0 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.694+0000 7fc7a2ffd700 1 -- 172.21.15.123:0/2880167721 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v46) v1 ==== 74+0+14055 (secure 0 0 0) 0x7fc7b00558d0 con 0x7fc7bc15ebe0 2023-10-06T04:33:15.696 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:15.696 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":46,"fsid":"1cea82d3-d2ef-4e20-8c3e-67296f7df92c","created":"2023-10-06T04:31:09.428979+0000","modified":"2023-10-06T04:33:11.444215+0000","last_up_change":"2023-10-06T04:31:20.874089+0000","last_in_change":"2023-10-06T04:31:12.090395+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"quincy","pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-10-06T04:31:20.921532+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"16","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}}},{"pool":2,"pool_name":"rbd","create_time":"2023-10-06T04:31:27.955056+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"19","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"6c52dd00-164f-43a0-9150-7dabdfa7fb5c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6808","nonce":119718},{"type":"v1","addr":"172.21.15.123:6809","nonce":119718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6810","nonce":119718},{"type":"v1","addr":"172.21.15.123:6811","nonce":119718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6814","nonce":119718},{"type":"v1","addr":"172.21.15.123:6815","nonce":119718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6812","nonce":119718},{"type":"v1","addr":"172.21.15.123:6813","nonce":119718}]},"public_addr":"172.21.15.123:6809/119718","cluster_addr":"172.21.15.123:6811/119718","heartbeat_back_addr":"172.21.15.123:6815/119718","heartbeat_front_addr":"172.21.15.123:6813/119718","state":["exists","up"]},{"osd":1,"uuid":"4616ab25-fd5a-44ae-a474-ee1400447513","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6816","nonce":119720},{"type":"v1","addr":"172.21.15.123:6817","nonce":119720}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6818","nonce":119720},{"type":"v1","addr":"172.21.15.123:6819","nonce":119720}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6822","nonce":119720},{"type":"v1","addr":"172.21.15.123:6823","nonce":119720}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6820","nonce":119720},{"type":"v1","addr":"172.21.15.123:6821","nonce":119720}]},"public_addr":"172.21.15.123:6817/119720","cluster_addr":"172.21.15.123:6819/119720","heartbeat_back_addr":"172.21.15.123:6823/119720","heartbeat_front_addr":"172.21.15.123:6821/119720","state":["exists","up"]},{"osd":2,"uuid":"7e970543-3fd8-472b-a1fa-bba74051dc6d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6824","nonce":119723},{"type":"v1","addr":"172.21.15.123:6825","nonce":119723}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6826","nonce":119723},{"type":"v1","addr":"172.21.15.123:6827","nonce":119723}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6830","nonce":119723},{"type":"v1","addr":"172.21.15.123:6831","nonce":119723}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6828","nonce":119723},{"type":"v1","addr":"172.21.15.123:6829","nonce":119723}]},"public_addr":"172.21.15.123:6825/119723","cluster_addr":"172.21.15.123:6827/119723","heartbeat_back_addr":"172.21.15.123:6831/119723","heartbeat_front_addr":"172.21.15.123:6829/119723","state":["exists","up"]},{"osd":3,"uuid":"48b5e737-f098-4e53-b19f-cce0011cf3b5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6800","nonce":119719},{"type":"v1","addr":"172.21.15.123:6801","nonce":119719}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6802","nonce":119719},{"type":"v1","addr":"172.21.15.123:6803","nonce":119719}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6806","nonce":119719},{"type":"v1","addr":"172.21.15.123:6807","nonce":119719}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6804","nonce":119719},{"type":"v1","addr":"172.21.15.123:6805","nonce":119719}]},"public_addr":"172.21.15.123:6801/119719","cluster_addr":"172.21.15.123:6803/119719","heartbeat_back_addr":"172.21.15.123:6807/119719","heartbeat_front_addr":"172.21.15.123:6805/119719","state":["exists","up"]},{"osd":4,"uuid":"013a1be3-b369-47e4-9dd9-93a620898e3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6808","nonce":119946},{"type":"v1","addr":"172.21.15.177:6809","nonce":119946}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6810","nonce":119946},{"type":"v1","addr":"172.21.15.177:6811","nonce":119946}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6814","nonce":119946},{"type":"v1","addr":"172.21.15.177:6815","nonce":119946}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6812","nonce":119946},{"type":"v1","addr":"172.21.15.177:6813","nonce":119946}]},"public_addr":"172.21.15.177:6809/119946","cluster_addr":"172.21.15.177:6811/119946","heartbeat_back_addr":"172.21.15.177:6815/119946","heartbeat_front_addr":"172.21.15.177:6813/119946","state":["exists","up"]},{"osd":5,"uuid":"2b7e928e-e086-4c2a-9d76-9ba1b050044c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6800","nonce":119947},{"type":"v1","addr":"172.21.15.177:6801","nonce":119947}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6802","nonce":119947},{"type":"v1","addr":"172.21.15.177:6803","nonce":119947}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6806","nonce":119947},{"type":"v1","addr":"172.21.15.177:6807","nonce":119947}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6804","nonce":119947},{"type":"v1","addr":"172.21.15.177:6805","nonce":119947}]},"public_addr":"172.21.15.177:6801/119947","cluster_addr":"172.21.15.177:6803/119947","heartbeat_back_addr":"172.21.15.177:6807/119947","heartbeat_front_addr":"172.21.15.177:6805/119947","state":["exists","up"]},{"osd":6,"uuid":"c42e4b64-fcb7-4e6f-b071-bd16175cd367","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6816","nonce":119976},{"type":"v1","addr":"172.21.15.177:6817","nonce":119976}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6818","nonce":119976},{"type":"v1","addr":"172.21.15.177:6819","nonce":119976}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6822","nonce":119976},{"type":"v1","addr":"172.21.15.177:6823","nonce":119976}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6820","nonce":119976},{"type":"v1","addr":"172.21.15.177:6821","nonce":119976}]},"public_addr":"172.21.15.177:6817/119976","cluster_addr":"172.21.15.177:6819/119976","heartbeat_back_addr":"172.21.15.177:6823/119976","heartbeat_front_addr":"172.21.15.177:6821/119976","state":["exists","up"]},{"osd":7,"uuid":"b4976457-ab03-4387-b58e-0ac8bf0dd489","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":40,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6824","nonce":119977},{"type":"v1","addr":"172.21.15.177:6825","nonce":119977}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6826","nonce":119977},{"type":"v1","addr":"172.21.15.177:6827","nonce":119977}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6830","nonce":119977},{"type":"v1","addr":"172.21.15.177:6831","nonce":119977}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6828","nonce":119977},{"type":"v1","addr":"172.21.15.177:6829","nonce":119977}]},"public_addr":"172.21.15.177:6825/119977","cluster_addr":"172.21.15.177:6827/119977","heartbeat_back_addr":"172.21.15.177:6831/119977","heartbeat_front_addr":"172.21.15.177:6829/119977","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.292682+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.311150+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.218837+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.197221+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.236703+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.233433+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.221871+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138320759226367,"old_weight":0,"last_purged_snaps_scrub":"2023-10-06T04:31:17.200984+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.123:6834/571369576":"2023-10-07T04:33:08.437166+0000","172.21.15.123:6836/571369576":"2023-10-07T04:33:08.437166+0000","172.21.15.177:6833/1099021797":"2023-10-07T04:32:33.579645+0000","172.21.15.177:6835/1099021797":"2023-10-07T04:32:33.579645+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-10-06T04:33:15.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc7a8066f30 msgr2=0x7fc7a80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc7a8066f30 0x7fc7a80693e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7fc7ac001ea0 tx=0x7fc7ac015000 comp rx=0 tx=0).stop 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7bc15ebe0 msgr2=0x7fc7bc159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7bc15ebe0 0x7fc7bc159b50 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fc7b00119c0 tx=0x7fc7b0008040 comp rx=0 tx=0).stop 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 shutdown_connections 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc7a8066f30 0x7fc7a80693e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc7bc06e170 0x7fc7bc06e5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc7a4003ae0 0x7fc7bc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 --2- 172.21.15.123:0/2880167721 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc7bc15ebe0 0x7fc7bc159b50 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 >> 172.21.15.123:0/2880167721 conn(0x7fc7bc06a1c0 msgr2=0x7fc7bc0b09a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:15.699 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 shutdown_connections 2023-10-06T04:33:15.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:15.696+0000 7fc7c16d7700 1 -- 172.21.15.123:0/2880167721 wait complete. 2023-10-06T04:33:16.711 INFO:tasks.ceph:Scrubbing osd.0 2023-10-06T04:33:16.711 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:16.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.844+0000 7f0031d72700 1 Processor -- start 2023-10-06T04:33:16.846 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.844+0000 7f0031d72700 1 -- start start 2023-10-06T04:33:16.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f0031d72700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 0x7f002c068a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.850 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f0031d72700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f002c06a150 0x7f002c0c6dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f0031d72700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f002c0685e0 con 0x7f002c0cc290 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f0031d72700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f002c068fb0 con 0x7f002c06a150 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f0031d72700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0632c0 con 0x7f002c159610 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002affd700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f002c0cc290 0x7f002c069a40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55418/0 (socket says 172.21.15.123:55418) 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002affd700 1 -- 172.21.15.123:0/991573498 learned_addr learned my addr 172.21.15.123:0/991573498 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002b7fe700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 0x7f002c068a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002bfff700 1 --2- 172.21.15.123:0/991573498 >> v2:172.21.15.177:3300/0 conn(0x7f002c06a150 0x7f002c0c6dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175494003 0 0) 0x7f002c0685e0 con 0x7f002c0cc290 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0014003570 con 0x7f002c0cc290 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3971555891 0 0) 0x7f0014003570 con 0x7f002c0cc290 2023-10-06T04:33:16.851 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 msgr2=0x7f002c068a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 0x7f002c068a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 >> v2:172.21.15.177:3300/0 conn(0x7f002c06a150 msgr2=0x7f002c0c6dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 --2- 172.21.15.123:0/991573498 >> v2:172.21.15.177:3300/0 conn(0x7f002c06a150 0x7f002c0c6dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0679d0 con 0x7f002c0cc290 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f00200015f0 con 0x7f002c0cc290 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 >> v1:172.21.15.123:6789/0 conn(0x7f002c0cc290 legacy=0x7f002c069a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0014003a60 0x7f0014003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 0x7f0014004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.852 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0014004d50 0x7f00140051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c068fb0 con 0x7f0014004d50 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0632c0 con 0x7f0014003a60 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.845+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f001400ba60 con 0x7f002c159610 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002bfff700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0014004d50 0x7f00140051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002b7fe700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0014003a60 0x7f0014003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002affd700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 0x7f0014004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002bfff700 1 -- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 msgr2=0x7f0014004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002bfff700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c159610 0x7f0014004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002bfff700 1 -- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0014003a60 msgr2=0x7f0014003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002bfff700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0014003a60 0x7f0014003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:16.853 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002bfff700 1 -- 172.21.15.123:0/991573498 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0014003570 con 0x7f0014004d50 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002bfff700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0014004d50 0x7f00140051c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f002c06a550 tx=0x7f002400b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7349874e738006b1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400ea00 con 0x7f0014004d50 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f002400f040 con 0x7f0014004d50 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.846+0000 7f002a7fc700 1 -- 172.21.15.123:0/991573498 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f002400ed90 con 0x7f0014004d50 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.847+0000 7f0031d72700 1 -- 172.21.15.123:0/991573498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0014004d50 msgr2=0x7f00140051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.847+0000 7f0031d72700 1 --2- 172.21.15.123:0/991573498 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0014004d50 0x7f00140051c0 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f002c06a550 tx=0x7f002400b040 comp rx=0 tx=0).stop 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.847+0000 7f002bfff700 1 -- 172.21.15.123:0/991573498 reap_dead start 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.847+0000 7f0031d72700 1 -- 172.21.15.123:0/991573498 shutdown_connections 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.847+0000 7f0031d72700 1 -- 172.21.15.123:0/991573498 >> 172.21.15.123:0/991573498 conn(0x7f002c0cd250 msgr2=0x7f002c0c91f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.847+0000 7f0031d72700 1 -- 172.21.15.123:0/991573498 shutdown_connections 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.847+0000 7f0031d72700 1 -- 172.21.15.123:0/991573498 wait complete. 2023-10-06T04:33:16.854 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.848+0000 7f0031d72700 1 Processor -- start 2023-10-06T04:33:16.855 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.848+0000 7f0031d72700 1 -- start start 2023-10-06T04:33:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f0031d72700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c154400 0x7f002c0c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f0031d72700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f002c0c3910 0x7f002c0b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.857 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f0031d72700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f002c1547d0 0x7f002c15f1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f0031d72700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0b1970 con 0x7f002c0c3910 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f0031d72700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0b1040 con 0x7f002c1547d0 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f0031d72700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f002c0b1320 con 0x7f002c154400 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f002c0c3910 0x7f002c0b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f002c0c3910 0x7f002c0b1e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41896/0 (socket says 172.21.15.123:41896) 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 -- 172.21.15.123:0/3989748935 learned_addr learned my addr 172.21.15.123:0/3989748935 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002b7fe700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c154400 0x7f002c0c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002bfff700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f002c1547d0 0x7f002c15f1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 -- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c154400 msgr2=0x7f002c0c4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c154400 0x7f002c0c4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:16.858 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 -- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f002c1547d0 msgr2=0x7f002c15f1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f002c1547d0 0x7f002c15f1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 -- 172.21.15.123:0/3989748935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f002c0b1600 con 0x7f002c0c3910 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002bfff700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f002c1547d0 0x7f002c15f1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002b7fe700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f002c154400 0x7f002c0c4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f002affd700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f002c0c3910 0x7f002c0b1e00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0020002bf0 tx=0x7f0020009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200072e0 con 0x7f002c0c3910 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0020007440 con 0x7f002c0c3910 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f002c0b0cf0 con 0x7f002c0c3910 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.849+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00200057f0 con 0x7f002c0c3910 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.850+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f002c0b2340 con 0x7f002c0c3910 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.850+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f002c069f90 con 0x7f002c0c3910 2023-10-06T04:33:16.859 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.851+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0020005c20 con 0x7f002c0c3910 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.851+0000 7f00137fe700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0018066f30 0x7f00180693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.851+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f0020084790 con 0x7f002c0c3910 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.851+0000 7f002b7fe700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0018066f30 0x7f00180693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.851+0000 7f00137fe700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f001806d440 0x7f001806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.851+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f001806ff00 con 0x7f001806d440 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.851+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0020086dd0 con 0x7f002c0c3910 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.852+0000 7f002bfff700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f001806d440 0x7f001806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.852+0000 7f002b7fe700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0018066f30 0x7f00180693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f001c006f00 tx=0x7f001c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.852+0000 7f002bfff700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f001806d440 0x7f001806f850 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:16.860 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.852+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== osd.0 v2:172.21.15.123:6808/119718 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f001806ff00 con 0x7f001806d440 2023-10-06T04:33:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.866+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 --> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f002c070f40 con 0x7f001806d440 2023-10-06T04:33:16.868 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.866+0000 7f00137fe700 1 -- 172.21.15.123:0/3989748935 <== osd.0 v2:172.21.15.123:6808/119718 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f002c070f40 con 0x7f001806d440 2023-10-06T04:33:16.869 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:16.869 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:16.869 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f001806d440 msgr2=0x7f001806f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6808/119718,v1:172.21.15.123:6809/119718] conn(0x7f001806d440 0x7f001806f850 crc :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0018066f30 msgr2=0x7f00180693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0018066f30 0x7f00180693e0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f001c006f00 tx=0x7f001c008040 comp rx=0 tx=0).stop 2023-10-06T04:33:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f002c0c3910 msgr2=0x7f002c0b1e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 --2- 172.21.15.123:0/3989748935 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f002c0c3910 0x7f002c0b1e00 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f0020002bf0 tx=0x7f0020009040 comp rx=0 tx=0).stop 2023-10-06T04:33:16.870 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f002bfff700 1 -- 172.21.15.123:0/3989748935 reap_dead start 2023-10-06T04:33:16.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 shutdown_connections 2023-10-06T04:33:16.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 >> 172.21.15.123:0/3989748935 conn(0x7f002c0cd250 msgr2=0x7f002c0c2640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:16.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 shutdown_connections 2023-10-06T04:33:16.871 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.867+0000 7f0031d72700 1 -- 172.21.15.123:0/3989748935 wait complete. 2023-10-06T04:33:16.881 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-10-06T04:33:16.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.996+0000 7fd002cb8700 1 Processor -- start 2023-10-06T04:33:16.999 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.996+0000 7fd002cb8700 1 -- start start 2023-10-06T04:33:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd002cb8700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcffc157140 0x7fcffc06bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd002cb8700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc06aa70 con 0x7fcffc0c3840 2023-10-06T04:33:17.001 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd002cb8700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcffc0cb810 con 0x7fcffc157510 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd002cb8700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcffc06a4b0 con 0x7fcffc157140 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd000a54700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcffc157140 0x7fcffc06bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd001255700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fcffc0c3840 0x7fcffc0cbca0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55430/0 (socket says 172.21.15.123:55430) 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd000a54700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fcffc157140 0x7fcffc06bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41904/0 (socket says 172.21.15.123:41904) 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd001255700 1 -- 172.21.15.123:0/2945254436 learned_addr learned my addr 172.21.15.123:0/2945254436 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd000a54700 1 -- 172.21.15.123:0/2945254436 >> v1:172.21.15.177:6789/0 conn(0x7fcffc157510 legacy=0x7fcffc0aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.997+0000 7fd000a54700 1 -- 172.21.15.123:0/2945254436 >> v1:172.21.15.123:6789/0 conn(0x7fcffc0c3840 legacy=0x7fcffc0cbca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fcffb7fe700 1 -- 172.21.15.123:0/2945254436 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4168660803 0 0) 0x7fcffc06aa70 con 0x7fcffc0c3840 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd000a54700 1 -- 172.21.15.123:0/2945254436 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcffc06a790 con 0x7fcffc157140 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd000a54700 1 --2- 172.21.15.123:0/2945254436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc06bf70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcff00098e0 tx=0x7fcff00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47e1533594b7be4b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.002 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fcffb7fe700 1 -- 172.21.15.123:0/2945254436 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff0009e80 con 0x7fcffc157140 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fcffb7fe700 1 -- 172.21.15.123:0/2945254436 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcff000b820 con 0x7fcffc157140 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fcffb7fe700 1 -- 172.21.15.123:0/2945254436 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcff000bba0 con 0x7fcffc157140 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd002cb8700 1 -- 172.21.15.123:0/2945254436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 msgr2=0x7fcffc06bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd002cb8700 1 --2- 172.21.15.123:0/2945254436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc06bf70 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7fcff00098e0 tx=0x7fcff00091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd002cb8700 1 -- 172.21.15.123:0/2945254436 shutdown_connections 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd002cb8700 1 --2- 172.21.15.123:0/2945254436 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc06bf70 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd002cb8700 1 -- 172.21.15.123:0/2945254436 >> 172.21.15.123:0/2945254436 conn(0x7fcffc0c9d00 msgr2=0x7fcffc06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd002cb8700 1 -- 172.21.15.123:0/2945254436 shutdown_connections 2023-10-06T04:33:17.003 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.998+0000 7fd002cb8700 1 -- 172.21.15.123:0/2945254436 wait complete. 2023-10-06T04:33:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:16.999+0000 7fd002cb8700 1 Processor -- start 2023-10-06T04:33:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.000+0000 7fd002cb8700 1 -- start start 2023-10-06T04:33:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.000+0000 7fd002cb8700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcffc0c3840 0x7fcffc0b2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.000+0000 7fd002cb8700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc0a98a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.000+0000 7fd002cb8700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcffc157510 0x7fcffc150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.000+0000 7fd002cb8700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0a9de0 con 0x7fcffc157140 2023-10-06T04:33:17.006 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.000+0000 7fd002cb8700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0c6f00 con 0x7fcffc0c3840 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.000+0000 7fd002cb8700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcffc0a9fd0 con 0x7fcffc157510 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc0a98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc0a98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41912/0 (socket says 172.21.15.123:41912) 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 -- 172.21.15.123:0/1559182739 learned_addr learned my addr 172.21.15.123:0/1559182739 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fd000a54700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcffc0c3840 0x7fcffc0b2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fd001255700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcffc157510 0x7fcffc150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 -- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcffc157510 msgr2=0x7fcffc150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcffc157510 0x7fcffc150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 -- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcffc0c3840 msgr2=0x7fcffc0b2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcffc0c3840 0x7fcffc0b2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.007 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 -- 172.21.15.123:0/1559182739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff0009420 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcffbfff700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc0a98a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800b5e0 tx=0x7fcfe8009120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcff8ff9700 1 -- 172.21.15.123:0/1559182739 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800f510 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcff8ff9700 1 -- 172.21.15.123:0/1559182739 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe800f670 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcffc0aa2b0 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fcff8ff9700 1 -- 172.21.15.123:0/1559182739 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe800fa30 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.001+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcffc0aa870 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.002+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcffc0c7090 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.003+0000 7fcff8ff9700 1 -- 172.21.15.123:0/1559182739 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fcfe801a400 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.003+0000 7fcff8ff9700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfe0066f30 0x7fcfe00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.003+0000 7fcff8ff9700 1 -- 172.21.15.123:0/1559182739 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fcfe8083e00 con 0x7fcffc157140 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.003+0000 7fd000a54700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfe0066f30 0x7fcfe00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.008 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.003+0000 7fd000a54700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfe0066f30 0x7fcfe00693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcff00030c0 tx=0x7fcff0016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.009 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.007+0000 7fcff8ff9700 1 -- 172.21.15.123:0/1559182739 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fcfe8051230 con 0x7fcffc157140 2023-10-06T04:33:17.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.187+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fcffc0cb4a0 con 0x7fcfe0066f30 2023-10-06T04:33:17.190 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.188+0000 7fcff8ff9700 1 -- 172.21.15.123:0/1559182739 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcffc0cb4a0 con 0x7fcfe0066f30 2023-10-06T04:33:17.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfe0066f30 msgr2=0x7fcfe00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.192 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfe0066f30 0x7fcfe00693e0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7fcff00030c0 tx=0x7fcff0016000 comp rx=0 tx=0).stop 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 msgr2=0x7fcffc0a98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc0a98a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7fcfe800b5e0 tx=0x7fcfe8009120 comp rx=0 tx=0).stop 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 shutdown_connections 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fcfe0066f30 0x7fcfe00693e0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fcffc157510 0x7fcffc150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fcffc0c3840 0x7fcffc0b2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 --2- 172.21.15.123:0/1559182739 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fcffc157140 0x7fcffc0a98a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 >> 172.21.15.123:0/1559182739 conn(0x7fcffc0c9d00 msgr2=0x7fcffc1529c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.193 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 shutdown_connections 2023-10-06T04:33:17.194 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.190+0000 7fd002cb8700 1 -- 172.21.15.123:0/1559182739 wait complete. 2023-10-06T04:33:17.194 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 0 to deep-scrub 2023-10-06T04:33:17.204 INFO:tasks.ceph:Scrubbing osd.1 2023-10-06T04:33:17.204 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:17.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.322+0000 7f34ba45b700 1 Processor -- start 2023-10-06T04:33:17.325 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.323+0000 7f34ba45b700 1 -- start start 2023-10-06T04:33:17.327 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.323+0000 7f34ba45b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34b40c5120 0x7f34b40cef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.323+0000 7f34ba45b700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f34b40b3140 0x7f34b40b3510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.323+0000 7f34ba45b700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b40c61c0 con 0x7f34b40cb360 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.323+0000 7f34ba45b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b40c55b0 con 0x7f34b40c5120 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.323+0000 7f34ba45b700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b40c5890 con 0x7f34b40b3140 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.323+0000 7f34b37fe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f34b40c5120 0x7f34b40cef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b3fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f34b40cb360 0x7f34b40cb730 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55434/0 (socket says 172.21.15.123:55434) 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b3fff700 1 -- 172.21.15.123:0/3129635558 learned_addr learned my addr 172.21.15.123:0/3129635558 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:17.328 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b89f8700 1 --2- 172.21.15.123:0/3129635558 >> v2:172.21.15.177:3300/0 conn(0x7f34b40b3140 0x7f34b40b3510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b37fe700 1 -- 172.21.15.123:0/3129635558 >> v2:172.21.15.177:3300/0 conn(0x7f34b40b3140 msgr2=0x7f34b40b3510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b37fe700 1 --2- 172.21.15.123:0/3129635558 >> v2:172.21.15.177:3300/0 conn(0x7f34b40b3140 0x7f34b40b3510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b37fe700 1 -- 172.21.15.123:0/3129635558 >> v1:172.21.15.123:6789/0 conn(0x7f34b40cb360 legacy=0x7f34b40cb730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b37fe700 1 -- 172.21.15.123:0/3129635558 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b40c5b70 con 0x7f34b40c5120 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b2ffd700 1 -- 172.21.15.123:0/3129635558 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296439928 0 0) 0x7f34b40c61c0 con 0x7f34b40cb360 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b89f8700 1 --2- 172.21.15.123:0/3129635558 >> v2:172.21.15.177:3300/0 conn(0x7f34b40b3140 0x7f34b40b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b37fe700 1 --2- 172.21.15.123:0/3129635558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40c5120 0x7f34b40cef60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f34b40cbe40 tx=0x7f34a40091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d109cdabf117dea1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b2ffd700 1 -- 172.21.15.123:0/3129635558 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400b740 con 0x7f34b40c5120 2023-10-06T04:33:17.329 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b2ffd700 1 -- 172.21.15.123:0/3129635558 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a400b8a0 con 0x7f34b40c5120 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.324+0000 7f34b2ffd700 1 -- 172.21.15.123:0/3129635558 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400bc40 con 0x7f34b40c5120 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 -- 172.21.15.123:0/3129635558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40c5120 msgr2=0x7f34b40cef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 --2- 172.21.15.123:0/3129635558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40c5120 0x7f34b40cef60 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f34b40cbe40 tx=0x7f34a40091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 -- 172.21.15.123:0/3129635558 shutdown_connections 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 --2- 172.21.15.123:0/3129635558 >> v2:172.21.15.177:3300/0 conn(0x7f34b40b3140 0x7f34b40b3510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 --2- 172.21.15.123:0/3129635558 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40c5120 0x7f34b40cef60 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 -- 172.21.15.123:0/3129635558 >> 172.21.15.123:0/3129635558 conn(0x7f34b4063a10 msgr2=0x7f34b40d0380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 -- 172.21.15.123:0/3129635558 shutdown_connections 2023-10-06T04:33:17.330 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.325+0000 7f34ba45b700 1 -- 172.21.15.123:0/3129635558 wait complete. 2023-10-06T04:33:17.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.326+0000 7f34ba45b700 1 Processor -- start 2023-10-06T04:33:17.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.326+0000 7f34ba45b700 1 -- start start 2023-10-06T04:33:17.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34ba45b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40b3140 0x7f34b4155fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34ba45b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34b40c5120 0x7f34b4149f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.334 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34ba45b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b40cb360 0x7f34b40cce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34ba45b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b415dc70 con 0x7f34b40b3140 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34ba45b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b4169720 con 0x7f34b40cb360 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34ba45b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40ca830 con 0x7f34b40c5120 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40b3140 0x7f34b4155fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40b3140 0x7f34b4155fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41928/0 (socket says 172.21.15.123:41928) 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 -- 172.21.15.123:0/1860548371 learned_addr learned my addr 172.21.15.123:0/1860548371 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b37fe700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34b40c5120 0x7f34b4149f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b89f8700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b40cb360 0x7f34b40cce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.335 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 -- 172.21.15.123:0/1860548371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34b40c5120 msgr2=0x7f34b4149f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f34b40c5120 0x7f34b4149f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 -- 172.21.15.123:0/1860548371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b40cb360 msgr2=0x7f34b40cce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f34b40cb360 0x7f34b40cce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 -- 172.21.15.123:0/1860548371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34a4009420 con 0x7f34b40b3140 2023-10-06T04:33:17.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34b3fff700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40b3140 0x7f34b4155fa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f34a800c5b0 tx=0x7f34a8001070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8005b30 con 0x7f34b40b3140 2023-10-06T04:33:17.336 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a8005c90 con 0x7f34b40b3140 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.327+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b40ca550 con 0x7f34b40b3140 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.328+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a8007300 con 0x7f34b40b3140 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.328+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b415d900 con 0x7f34b40b3140 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.328+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f34b406a250 con 0x7f34b40b3140 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.329+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f34a801b3b0 con 0x7f34b40b3140 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.329+0000 7f349bfff700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34a0066f30 0x7f34a00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.329+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f34a8084060 con 0x7f34b40b3140 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.329+0000 7f34b37fe700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34a0066f30 0x7f34a00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.337 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.329+0000 7f349bfff700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f34a006d440 0x7f34a006f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.329+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f34a006ff00 con 0x7f34a006d440 2023-10-06T04:33:17.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.329+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f34a8084430 con 0x7f34b40b3140 2023-10-06T04:33:17.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.330+0000 7f34b89f8700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f34a006d440 0x7f34a006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.330+0000 7f34b37fe700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34a0066f30 0x7f34a00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34a40093f0 tx=0x7f34a4017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.330+0000 7f34b89f8700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f34a006d440 0x7f34a006f850 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.338 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.330+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== osd.1 v2:172.21.15.123:6816/119720 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f34a006ff00 con 0x7f34a006d440 2023-10-06T04:33:17.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 --> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f34b40c5e50 con 0x7f34a006d440 2023-10-06T04:33:17.346 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f349bfff700 1 -- 172.21.15.123:0/1860548371 <== osd.1 v2:172.21.15.123:6816/119720 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f34b40c5e50 con 0x7f34a006d440 2023-10-06T04:33:17.347 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:17.347 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:17.347 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f34a006d440 msgr2=0x7f34a006f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6816/119720,v1:172.21.15.123:6817/119720] conn(0x7f34a006d440 0x7f34a006f850 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34a0066f30 msgr2=0x7f34a00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.348 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f34a0066f30 0x7f34a00693e0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f34a40093f0 tx=0x7f34a4017000 comp rx=0 tx=0).stop 2023-10-06T04:33:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40b3140 msgr2=0x7f34b4155fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 --2- 172.21.15.123:0/1860548371 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f34b40b3140 0x7f34b4155fa0 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f34a800c5b0 tx=0x7f34a8001070 comp rx=0 tx=0).stop 2023-10-06T04:33:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34b89f8700 1 -- 172.21.15.123:0/1860548371 reap_dead start 2023-10-06T04:33:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 shutdown_connections 2023-10-06T04:33:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.344+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 >> 172.21.15.123:0/1860548371 conn(0x7f34b4063a10 msgr2=0x7f34b4158300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.345+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 shutdown_connections 2023-10-06T04:33:17.349 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.345+0000 7f34ba45b700 1 -- 172.21.15.123:0/1860548371 wait complete. 2023-10-06T04:33:17.358 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-10-06T04:33:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.475+0000 7f38a756b700 1 Processor -- start 2023-10-06T04:33:17.477 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.475+0000 7f38a756b700 1 -- start start 2023-10-06T04:33:17.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a756b700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38a01560e0 0x7f38a006b010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.480 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a756b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a0068770 0x7f38a0068be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a756b700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38a0069120 con 0x7f38a006e250 2023-10-06T04:33:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a756b700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38a006f1f0 con 0x7f38a01560e0 2023-10-06T04:33:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a756b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a006ab80 con 0x7f38a0068770 2023-10-06T04:33:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38a01560e0 0x7f38a006b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f38a01560e0 0x7f38a006b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41930/0 (socket says 172.21.15.123:41930) 2023-10-06T04:33:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 -- 172.21.15.123:0/2796106218 learned_addr learned my addr 172.21.15.123:0/2796106218 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:17.481 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 -- 172.21.15.123:0/2796106218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a0068770 msgr2=0x7f38a0068be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a4b06700 1 --2- 172.21.15.123:0/2796106218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a0068770 0x7f38a0068be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 --2- 172.21.15.123:0/2796106218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a0068770 0x7f38a0068be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 -- 172.21.15.123:0/2796106218 >> v1:172.21.15.177:6789/0 conn(0x7f38a006e250 legacy=0x7f38a006e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 -- 172.21.15.123:0/2796106218 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38a006ef10 con 0x7f38a01560e0 2023-10-06T04:33:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a4b06700 1 --2- 172.21.15.123:0/2796106218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a0068770 0x7f38a0068be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:17.482 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f38a5307700 1 --2- 172.21.15.123:0/2796106218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a01560e0 0x7f38a006b010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f389c0099b0 tx=0x7f389c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d6dc98d2fe111d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f3897fff700 1 -- 172.21.15.123:0/2796106218 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00b820 con 0x7f38a01560e0 2023-10-06T04:33:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.476+0000 7f3897fff700 1 -- 172.21.15.123:0/2796106218 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c00b980 con 0x7f38a01560e0 2023-10-06T04:33:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f3897fff700 1 -- 172.21.15.123:0/2796106218 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c00bd20 con 0x7f38a01560e0 2023-10-06T04:33:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 -- 172.21.15.123:0/2796106218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a01560e0 msgr2=0x7f38a006b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 --2- 172.21.15.123:0/2796106218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a01560e0 0x7f38a006b010 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f389c0099b0 tx=0x7f389c009250 comp rx=0 tx=0).stop 2023-10-06T04:33:17.483 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 -- 172.21.15.123:0/2796106218 shutdown_connections 2023-10-06T04:33:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 --2- 172.21.15.123:0/2796106218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a01560e0 0x7f38a006b010 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 --2- 172.21.15.123:0/2796106218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a0068770 0x7f38a0068be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 -- 172.21.15.123:0/2796106218 >> 172.21.15.123:0/2796106218 conn(0x7f38a00cd010 msgr2=0x7f38a00cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 -- 172.21.15.123:0/2796106218 shutdown_connections 2023-10-06T04:33:17.484 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.477+0000 7f38a756b700 1 -- 172.21.15.123:0/2796106218 wait complete. 2023-10-06T04:33:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.479+0000 7f38a756b700 1 Processor -- start 2023-10-06T04:33:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.479+0000 7f38a756b700 1 -- start start 2023-10-06T04:33:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a756b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a0068770 0x7f38a014e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.487 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a756b700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a006e250 0x7f38a00ae440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a756b700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a01560e0 0x7f38a0155330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a756b700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00aef40 con 0x7f38a0068770 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a756b700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c5280 con 0x7f38a006e250 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a756b700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38a00c5560 con 0x7f38a01560e0 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a0068770 0x7f38a014e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a0068770 0x7f38a014e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41934/0 (socket says 172.21.15.123:41934) 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 -- 172.21.15.123:0/3600357011 learned_addr learned my addr 172.21.15.123:0/3600357011 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:17.488 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a4b06700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a006e250 0x7f38a00ae440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5b08700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a01560e0 0x7f38a0155330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 -- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a01560e0 msgr2=0x7f38a0155330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a01560e0 0x7f38a0155330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 -- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a006e250 msgr2=0x7f38a00ae440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a006e250 0x7f38a00ae440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 -- 172.21.15.123:0/3600357011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f389c0094f0 con 0x7f38a0068770 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5b08700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a01560e0 0x7f38a0155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:17.489 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a4b06700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a006e250 0x7f38a00ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a5307700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a0068770 0x7f38a014e760 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f389c009290 tx=0x7f389c016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38957fa700 1 -- 172.21.15.123:0/3600357011 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c01b470 con 0x7f38a0068770 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38957fa700 1 -- 172.21.15.123:0/3600357011 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f389c01b5d0 con 0x7f38a0068770 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38a00c5840 con 0x7f38a0068770 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.480+0000 7f38957fa700 1 -- 172.21.15.123:0/3600357011 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f389c01b990 con 0x7f38a0068770 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.481+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38a014eca0 con 0x7f38a0068770 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.482+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38a006d2b0 con 0x7f38a0068770 2023-10-06T04:33:17.490 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.482+0000 7f38957fa700 1 -- 172.21.15.123:0/3600357011 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f389c01c450 con 0x7f38a0068770 2023-10-06T04:33:17.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.482+0000 7f38957fa700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3884066f30 0x7f38840693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.482+0000 7f38a4b06700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3884066f30 0x7f38840693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.482+0000 7f38957fa700 1 -- 172.21.15.123:0/3600357011 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f389c086110 con 0x7f38a0068770 2023-10-06T04:33:17.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.483+0000 7f38a4b06700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3884066f30 0x7f38840693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38a006f8d0 tx=0x7f3890008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.491 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.487+0000 7f38957fa700 1 -- 172.21.15.123:0/3600357011 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f389c0534c0 con 0x7f38a0068770 2023-10-06T04:33:17.670 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.668+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f38a00c53f0 con 0x7f3884066f30 2023-10-06T04:33:17.671 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.668+0000 7f38957fa700 1 -- 172.21.15.123:0/3600357011 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f38a00c53f0 con 0x7f3884066f30 2023-10-06T04:33:17.675 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3884066f30 msgr2=0x7f38840693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.675 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3884066f30 0x7f38840693e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38a006f8d0 tx=0x7f3890008040 comp rx=0 tx=0).stop 2023-10-06T04:33:17.675 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a0068770 msgr2=0x7f38a014e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.675 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a0068770 0x7f38a014e760 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f389c009290 tx=0x7f389c016000 comp rx=0 tx=0).stop 2023-10-06T04:33:17.675 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 shutdown_connections 2023-10-06T04:33:17.675 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3884066f30 0x7f38840693e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f38a0068770 0x7f38a014e760 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f38a01560e0 0x7f38a0155330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 --2- 172.21.15.123:0/3600357011 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f38a006e250 0x7f38a00ae440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 >> 172.21.15.123:0/3600357011 conn(0x7f38a00cd010 msgr2=0x7f38a00c78e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 shutdown_connections 2023-10-06T04:33:17.676 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.670+0000 7f38a756b700 1 -- 172.21.15.123:0/3600357011 wait complete. 2023-10-06T04:33:17.676 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 1 to deep-scrub 2023-10-06T04:33:17.684 INFO:tasks.ceph:Scrubbing osd.2 2023-10-06T04:33:17.685 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:17.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.803+0000 7f4fd4183700 1 Processor -- start 2023-10-06T04:33:17.806 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.803+0000 7f4fd4183700 1 -- start start 2023-10-06T04:33:17.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.804+0000 7f4fd4183700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 0x7f4fcc06d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.804+0000 7f4fd4183700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc06d3b0 con 0x7f4fcc159550 2023-10-06T04:33:17.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.804+0000 7f4fd4183700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4fcc0ca7b0 con 0x7f4fcc159920 2023-10-06T04:33:17.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.804+0000 7f4fd4183700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc06dd80 con 0x7f4fcc06bb60 2023-10-06T04:33:17.814 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.804+0000 7f4fd1f1f700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f4fcc159550 0x7f4fcc06b450 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55436/0 (socket says 172.21.15.123:55436) 2023-10-06T04:33:17.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.804+0000 7f4fd1f1f700 1 -- 172.21.15.123:0/1029330290 learned_addr learned my addr 172.21.15.123:0/1029330290 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:17.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd171e700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 0x7f4fcc06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269741018 0 0) 0x7f4fcc06d3b0 con 0x7f4fcc159550 2023-10-06T04:33:17.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fb8003570 con 0x7f4fcc159550 2023-10-06T04:33:17.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3709144693 0 0) 0x7f4fcc0ca7b0 con 0x7f4fcc159920 2023-10-06T04:33:17.815 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4fcc06d3b0 con 0x7f4fcc159920 2023-10-06T04:33:17.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4219945240 0 0) 0x7f4fb8003570 con 0x7f4fcc159550 2023-10-06T04:33:17.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 msgr2=0x7f4fcc06d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 0x7f4fcc06d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.816 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 >> v1:172.21.15.177:6789/0 conn(0x7f4fcc159920 legacy=0x7f4fcc159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc15a5a0 con 0x7f4fcc159550 2023-10-06T04:33:17.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f4fbc001470 con 0x7f4fcc159550 2023-10-06T04:33:17.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 >> v1:172.21.15.123:6789/0 conn(0x7f4fcc159550 legacy=0x7f4fcc06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.817 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8003c20 0x7f4fb8004010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 0x7f4fb8004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc06dd80 con 0x7f4fb8004d50 2023-10-06T04:33:17.818 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb800bbf0 con 0x7f4fb8003c20 2023-10-06T04:33:17.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fb800bdb0 con 0x7f4fcc06bb60 2023-10-06T04:33:17.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd2720700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd1f1f700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8003c20 0x7f4fb8004010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.819 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd171e700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 0x7f4fb8004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd2720700 1 -- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 msgr2=0x7f4fb8004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd2720700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc06bb60 0x7f4fb8004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd2720700 1 -- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8003c20 msgr2=0x7f4fb8004010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd2720700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8003c20 0x7f4fb8004010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.805+0000 7f4fd2720700 1 -- 172.21.15.123:0/1029330290 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fb8003570 con 0x7f4fb8004d50 2023-10-06T04:33:17.820 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd2720700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40020f0 tx=0x7f4fc40017a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a48fc1332189dd1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400c8b0 con 0x7f4fb8004d50 2023-10-06T04:33:17.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc400ca10 con 0x7f4fb8004d50 2023-10-06T04:33:17.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd0f1d700 1 -- 172.21.15.123:0/1029330290 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc400cdb0 con 0x7f4fb8004d50 2023-10-06T04:33:17.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd4183700 1 -- 172.21.15.123:0/1029330290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb8004d50 msgr2=0x7f4fb80051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd4183700 1 --2- 172.21.15.123:0/1029330290 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fb8004d50 0x7f4fb80051c0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f4fc40020f0 tx=0x7f4fc40017a0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.821 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd2720700 1 -- 172.21.15.123:0/1029330290 reap_dead start 2023-10-06T04:33:17.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd4183700 1 -- 172.21.15.123:0/1029330290 shutdown_connections 2023-10-06T04:33:17.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd4183700 1 -- 172.21.15.123:0/1029330290 >> 172.21.15.123:0/1029330290 conn(0x7f4fcc0ce0c0 msgr2=0x7f4fcc0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd4183700 1 -- 172.21.15.123:0/1029330290 shutdown_connections 2023-10-06T04:33:17.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.806+0000 7f4fd4183700 1 -- 172.21.15.123:0/1029330290 wait complete. 2023-10-06T04:33:17.822 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.807+0000 7f4fd4183700 1 Processor -- start 2023-10-06T04:33:17.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.807+0000 7f4fd4183700 1 -- start start 2023-10-06T04:33:17.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd4183700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8004d50 0x7f4fcc16ad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd4183700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fcc0c3900 0x7f4fcc15b700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd4183700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc15bc40 0x7f4fcc155390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd4183700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc15c0b0 con 0x7f4fcc0c3900 2023-10-06T04:33:17.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd4183700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc154f00 con 0x7f4fb8004d50 2023-10-06T04:33:17.823 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd4183700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4fcc0c3ec0 con 0x7f4fcc15bc40 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd171e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fcc0c3900 0x7f4fcc15b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd171e700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fcc0c3900 0x7f4fcc15b700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41942/0 (socket says 172.21.15.123:41942) 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd171e700 1 -- 172.21.15.123:0/4100290066 learned_addr learned my addr 172.21.15.123:0/4100290066 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd2720700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc15bc40 0x7f4fcc155390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.808+0000 7f4fd1f1f700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8004d50 0x7f4fcc16ad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd171e700 1 -- 172.21.15.123:0/4100290066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc15bc40 msgr2=0x7f4fcc155390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd171e700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f4fcc15bc40 0x7f4fcc155390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd171e700 1 -- 172.21.15.123:0/4100290066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8004d50 msgr2=0x7f4fcc16ad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd171e700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f4fb8004d50 0x7f4fcc16ad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd171e700 1 -- 172.21.15.123:0/4100290066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4fcc0c41a0 con 0x7f4fcc0c3900 2023-10-06T04:33:17.824 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd171e700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fcc0c3900 0x7f4fcc15b700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006ac0 tx=0x7f4fc80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800f940 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4fc800faa0 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4fc800fe20 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4fcc0c4480 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.809+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4fcc1558d0 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.810+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4fcc06e690 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.810+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f4fc8014650 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fc1ffb700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4fb0066f30 0x7f4fb00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f4fc8018030 con 0x7f4fcc0c3900 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fd1f1f700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4fb0066f30 0x7f4fb00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.825 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fc1ffb700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f4fb006d440 0x7f4fb006f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4fb006ff00 con 0x7f4fb006d440 2023-10-06T04:33:17.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4fc8089240 con 0x7f4fcc0c3900 2023-10-06T04:33:17.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fd2720700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f4fb006d440 0x7f4fb006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fd1f1f700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4fb0066f30 0x7f4fb00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc001200 tx=0x7f4fbc001df0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.811+0000 7f4fd2720700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f4fb006d440 0x7f4fb006f850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.826 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.812+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== osd.2 v2:172.21.15.123:6824/119723 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f4fb006ff00 con 0x7f4fb006d440 2023-10-06T04:33:17.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.825+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 --> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4fcc15a880 con 0x7f4fb006d440 2023-10-06T04:33:17.827 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.825+0000 7f4fc1ffb700 1 -- 172.21.15.123:0/4100290066 <== osd.2 v2:172.21.15.123:6824/119723 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4fcc15a880 con 0x7f4fb006d440 2023-10-06T04:33:17.828 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:17.828 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:17.828 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:17.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f4fb006d440 msgr2=0x7f4fb006f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6824/119723,v1:172.21.15.123:6825/119723] conn(0x7f4fb006d440 0x7f4fb006f850 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4fb0066f30 msgr2=0x7f4fb00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f4fb0066f30 0x7f4fb00693e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc001200 tx=0x7f4fbc001df0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fcc0c3900 msgr2=0x7f4fcc15b700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.829 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 --2- 172.21.15.123:0/4100290066 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f4fcc0c3900 0x7f4fcc15b700 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f4fc8006ac0 tx=0x7f4fc80091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd2720700 1 -- 172.21.15.123:0/4100290066 reap_dead start 2023-10-06T04:33:17.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 shutdown_connections 2023-10-06T04:33:17.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 >> 172.21.15.123:0/4100290066 conn(0x7f4fcc0ce0c0 msgr2=0x7f4fcc15c390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 shutdown_connections 2023-10-06T04:33:17.830 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.826+0000 7f4fd4183700 1 -- 172.21.15.123:0/4100290066 wait complete. 2023-10-06T04:33:17.839 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-10-06T04:33:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.956+0000 7f872b7dd700 1 Processor -- start 2023-10-06T04:33:17.959 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f872b7dd700 1 -- start start 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f872b7dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f87240c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f872b7dd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87240b0420 0x7f872415a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f872b7dd700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f872415cfa0 con 0x7f87240b0050 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f872b7dd700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87240cedc0 con 0x7f87240b0420 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f872b7dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8724069780 con 0x7f872406bb80 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f8729d7a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87240b0420 0x7f872415a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.957+0000 7f8729d7a700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f87240b0420 0x7f872415a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41952/0 (socket says 172.21.15.123:41952) 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f8729d7a700 1 -- 172.21.15.123:0/95419128 learned_addr learned my addr 172.21.15.123:0/95419128 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:17.962 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f8728d78700 1 --2- 172.21.15.123:0/95419128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f87240c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f8729d7a700 1 -- 172.21.15.123:0/95419128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 msgr2=0x7f87240c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f8729d7a700 1 --2- 172.21.15.123:0/95419128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f87240c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f8729d7a700 1 -- 172.21.15.123:0/95419128 >> v1:172.21.15.177:6789/0 conn(0x7f87240b0050 legacy=0x7f872406b470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f8729d7a700 1 -- 172.21.15.123:0/95419128 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f872415a910 con 0x7f87240b0420 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f8729d7a700 1 --2- 172.21.15.123:0/95419128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0420 0x7f872415a3d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f871c009f20 tx=0x7f871c009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cc87f2b8b09bce7 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f871bfff700 1 -- 172.21.15.123:0/95419128 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c00e070 con 0x7f87240b0420 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f871bfff700 1 -- 172.21.15.123:0/95419128 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f871c00ba80 con 0x7f87240b0420 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.958+0000 7f871bfff700 1 -- 172.21.15.123:0/95419128 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f871c0092e0 con 0x7f87240b0420 2023-10-06T04:33:17.963 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 -- 172.21.15.123:0/95419128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0420 msgr2=0x7f872415a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 --2- 172.21.15.123:0/95419128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0420 0x7f872415a3d0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f871c009f20 tx=0x7f871c009840 comp rx=0 tx=0).stop 2023-10-06T04:33:17.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 -- 172.21.15.123:0/95419128 shutdown_connections 2023-10-06T04:33:17.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 --2- 172.21.15.123:0/95419128 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0420 0x7f872415a3d0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 --2- 172.21.15.123:0/95419128 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f87240c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 -- 172.21.15.123:0/95419128 >> 172.21.15.123:0/95419128 conn(0x7f8724068130 msgr2=0x7f87240ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:17.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 -- 172.21.15.123:0/95419128 shutdown_connections 2023-10-06T04:33:17.964 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.959+0000 7f872b7dd700 1 -- 172.21.15.123:0/95419128 wait complete. 2023-10-06T04:33:17.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.960+0000 7f872b7dd700 1 Processor -- start 2023-10-06T04:33:17.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.960+0000 7f872b7dd700 1 -- start start 2023-10-06T04:33:17.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f8724142950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.967 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0050 0x7f87240b3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87240b0420 0x7f872413fc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87241424c0 con 0x7f87240b0050 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8724142e90 con 0x7f87240b0420 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f872416de40 con 0x7f872406bb80 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0050 0x7f87240b3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0050 0x7f87240b3600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41962/0 (socket says 172.21.15.123:41962) 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 -- 172.21.15.123:0/333512137 learned_addr learned my addr 172.21.15.123:0/333512137 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 -- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 msgr2=0x7f8724142950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8729579700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f8724142950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.968 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8729d7a700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87240b0420 0x7f872413fc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f8724142950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 -- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87240b0420 msgr2=0x7f872413fc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87240b0420 0x7f872413fc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 -- 172.21.15.123:0/333512137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f871c009ae0 con 0x7f87240b0050 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8729579700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f8724142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8729d7a700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87240b0420 0x7f872413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f8728d78700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0050 0x7f87240b3600 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f872000d890 tx=0x7f8720008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.969 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f87197fa700 1 -- 172.21.15.123:0/333512137 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000b600 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f87197fa700 1 -- 172.21.15.123:0/333512137 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f872000b760 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8724141e70 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f87197fa700 1 -- 172.21.15.123:0/333512137 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f872000bae0 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.961+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87240b3b40 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.962+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8724142630 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.963+0000 7f87197fa700 1 -- 172.21.15.123:0/333512137 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f872001e460 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.963+0000 7f87197fa700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.963+0000 7f87197fa700 1 -- 172.21.15.123:0/333512137 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f8720084aa0 con 0x7f87240b0050 2023-10-06T04:33:17.970 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.963+0000 7f8729579700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:17.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.964+0000 7f8729579700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8708066f30 0x7f87080693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87240b3f90 tx=0x7f8714009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:17.971 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:17.967+0000 7f87197fa700 1 -- 172.21.15.123:0/333512137 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8720051e50 con 0x7f87240b0050 2023-10-06T04:33:18.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.149+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f87240c4eb0 con 0x7f8708066f30 2023-10-06T04:33:18.151 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.149+0000 7f87197fa700 1 -- 172.21.15.123:0/333512137 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f87240c4eb0 con 0x7f8708066f30 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8708066f30 msgr2=0x7f87080693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8708066f30 0x7f87080693e0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f87240b3f90 tx=0x7f8714009040 comp rx=0 tx=0).stop 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0050 msgr2=0x7f87240b3600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0050 0x7f87240b3600 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f872000d890 tx=0x7f8720008040 comp rx=0 tx=0).stop 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 shutdown_connections 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8708066f30 0x7f87080693e0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f87240b0420 0x7f872413fc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f872406bb80 0x7f8724142950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.154 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 --2- 172.21.15.123:0/333512137 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f87240b0050 0x7f87240b3600 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 >> 172.21.15.123:0/333512137 conn(0x7f8724068130 msgr2=0x7f872415aad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 shutdown_connections 2023-10-06T04:33:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.151+0000 7f872b7dd700 1 -- 172.21.15.123:0/333512137 wait complete. 2023-10-06T04:33:18.155 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 2 to deep-scrub 2023-10-06T04:33:18.165 INFO:tasks.ceph:Scrubbing osd.3 2023-10-06T04:33:18.165 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:18.287 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.285+0000 7ff553ea1700 1 Processor -- start 2023-10-06T04:33:18.288 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.285+0000 7ff553ea1700 1 -- start start 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff553ea1700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff54c06bb60 0x7ff54c06d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff553ea1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159920 0x7ff54c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff553ea1700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff54c06d3b0 con 0x7ff54c159550 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff553ea1700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff54c06dd80 con 0x7ff54c06bb60 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff553ea1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0ca7b0 con 0x7ff54c159920 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff55143c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff54c06bb60 0x7ff54c06d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff55143c700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7ff54c06bb60 0x7ff54c06d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41974/0 (socket says 172.21.15.123:41974) 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff55143c700 1 -- 172.21.15.123:0/431082913 learned_addr learned my addr 172.21.15.123:0/431082913 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:18.290 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.286+0000 7ff55243e700 1 --2- 172.21.15.123:0/431082913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159920 0x7ff54c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff55143c700 1 -- 172.21.15.123:0/431082913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159920 msgr2=0x7ff54c159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff55143c700 1 --2- 172.21.15.123:0/431082913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159920 0x7ff54c159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff55143c700 1 -- 172.21.15.123:0/431082913 >> v1:172.21.15.177:6789/0 conn(0x7ff54c159550 legacy=0x7ff54c06b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff55143c700 1 -- 172.21.15.123:0/431082913 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff54c15a300 con 0x7ff54c06bb60 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff55143c700 1 --2- 172.21.15.123:0/431082913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c06d840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0099b0 tx=0x7ff53c009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07320cfab1510c9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff550c3b700 1 -- 172.21.15.123:0/431082913 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00b860 con 0x7ff54c06bb60 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff550c3b700 1 -- 172.21.15.123:0/431082913 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff53c00b9c0 con 0x7ff54c06bb60 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff550c3b700 1 -- 172.21.15.123:0/431082913 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff53c00bd80 con 0x7ff54c06bb60 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff553ea1700 1 -- 172.21.15.123:0/431082913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 msgr2=0x7ff54c06d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff553ea1700 1 --2- 172.21.15.123:0/431082913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c06d840 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0099b0 tx=0x7ff53c009250 comp rx=0 tx=0).stop 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff553ea1700 1 -- 172.21.15.123:0/431082913 shutdown_connections 2023-10-06T04:33:18.291 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff553ea1700 1 --2- 172.21.15.123:0/431082913 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159920 0x7ff54c159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff553ea1700 1 --2- 172.21.15.123:0/431082913 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c06d840 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.287+0000 7ff553ea1700 1 -- 172.21.15.123:0/431082913 >> 172.21.15.123:0/431082913 conn(0x7ff54c0ce0c0 msgr2=0x7ff54c0ce4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.288+0000 7ff553ea1700 1 -- 172.21.15.123:0/431082913 shutdown_connections 2023-10-06T04:33:18.292 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.288+0000 7ff553ea1700 1 -- 172.21.15.123:0/431082913 wait complete. 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.289+0000 7ff553ea1700 1 Processor -- start 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.289+0000 7ff553ea1700 1 -- start start 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff553ea1700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff553ea1700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159550 0x7ff54c16a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff553ea1700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff54c159920 0x7ff54c1571d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff553ea1700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0c81a0 con 0x7ff54c06bb60 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff553ea1700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c156d40 con 0x7ff54c159920 2023-10-06T04:33:18.296 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff553ea1700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff54c0c7c60 con 0x7ff54c159550 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41978/0 (socket says 172.21.15.123:41978) 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 -- 172.21.15.123:0/3847569081 learned_addr learned my addr 172.21.15.123:0/3847569081 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff55143c700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159550 0x7ff54c16a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff55243e700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff54c159920 0x7ff54c1571d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 -- 172.21.15.123:0/3847569081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159550 msgr2=0x7ff54c16a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff54c159550 0x7ff54c16a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 -- 172.21.15.123:0/3847569081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff54c159920 msgr2=0x7ff54c1571d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff54c159920 0x7ff54c1571d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.290+0000 7ff551c3d700 1 -- 172.21.15.123:0/3847569081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff53c0094f0 con 0x7ff54c06bb60 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.291+0000 7ff551c3d700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c154580 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c5c0 tx=0x7ff54800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.297 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.291+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548011590 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.291+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff548011710 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.291+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff54c16a920 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.291+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff548011ad0 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.291+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff54c1566f0 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.292+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff54c06e690 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.292+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff54801b3b0 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.292+0000 7ff541ffb700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff530066f60 0x7ff530069410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7ff5480839d0 con 0x7ff54c06bb60 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff55143c700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff530066f60 0x7ff530069410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff541ffb700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7ff53006d470 0x7ff53006f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff53006ff30 con 0x7ff53006d470 2023-10-06T04:33:18.298 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff548088240 con 0x7ff54c06bb60 2023-10-06T04:33:18.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff55243e700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7ff53006d470 0x7ff53006f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff55143c700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff530066f60 0x7ff530069410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0099b0 tx=0x7ff53c017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.293+0000 7ff55243e700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7ff53006d470 0x7ff53006f880 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.299 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.294+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== osd.3 v2:172.21.15.123:6800/119719 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff53006ff30 con 0x7ff53006d470 2023-10-06T04:33:18.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.307+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 --> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff54c15a5a0 con 0x7ff53006d470 2023-10-06T04:33:18.309 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.307+0000 7ff541ffb700 1 -- 172.21.15.123:0/3847569081 <== osd.3 v2:172.21.15.123:6800/119719 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff54c15a5a0 con 0x7ff53006d470 2023-10-06T04:33:18.310 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:18.310 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:18.310 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:18.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7ff53006d470 msgr2=0x7ff53006f880 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6800/119719,v1:172.21.15.123:6801/119719] conn(0x7ff53006d470 0x7ff53006f880 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff530066f60 msgr2=0x7ff530069410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff530066f60 0x7ff530069410 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7ff53c0099b0 tx=0x7ff53c017000 comp rx=0 tx=0).stop 2023-10-06T04:33:18.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 msgr2=0x7ff54c154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.311 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 --2- 172.21.15.123:0/3847569081 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff54c06bb60 0x7ff54c154580 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7ff54800c5c0 tx=0x7ff54800c980 comp rx=0 tx=0).stop 2023-10-06T04:33:18.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff55243e700 1 -- 172.21.15.123:0/3847569081 reap_dead start 2023-10-06T04:33:18.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 shutdown_connections 2023-10-06T04:33:18.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 >> 172.21.15.123:0/3847569081 conn(0x7ff54c0ce0c0 msgr2=0x7ff54c0cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 shutdown_connections 2023-10-06T04:33:18.312 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.308+0000 7ff553ea1700 1 -- 172.21.15.123:0/3847569081 wait complete. 2023-10-06T04:33:18.321 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-10-06T04:33:18.438 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.436+0000 7fded7fc6700 1 Processor -- start 2023-10-06T04:33:18.439 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.436+0000 7fded7fc6700 1 -- start start 2023-10-06T04:33:18.441 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded7fc6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c8680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded7fc6700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fded00b0420 0x7fded015a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded7fc6700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fded006b470 0x7fded006b8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded7fc6700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded00cedc0 con 0x7fded006b470 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded7fc6700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fded006be30 con 0x7fded00b0420 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded7fc6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded015cfa0 con 0x7fded00b0050 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded6563700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fded006b470 0x7fded006b8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded6563700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fded006b470 0x7fded006b8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41992/0 (socket says 172.21.15.123:41992) 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded6563700 1 -- 172.21.15.123:0/1973367218 learned_addr learned my addr 172.21.15.123:0/1973367218 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:18.442 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded5561700 1 --2- 172.21.15.123:0/1973367218 >> v2:172.21.15.177:3300/0 conn(0x7fded00b0420 0x7fded015a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.437+0000 7fded5d62700 1 --2- 172.21.15.123:0/1973367218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c8680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded6563700 1 -- 172.21.15.123:0/1973367218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 msgr2=0x7fded00c8680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded6563700 1 --2- 172.21.15.123:0/1973367218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c8680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded6563700 1 -- 172.21.15.123:0/1973367218 >> v2:172.21.15.177:3300/0 conn(0x7fded00b0420 msgr2=0x7fded015a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded6563700 1 --2- 172.21.15.123:0/1973367218 >> v2:172.21.15.177:3300/0 conn(0x7fded00b0420 0x7fded015a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded6563700 1 -- 172.21.15.123:0/1973367218 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fded015a910 con 0x7fded006b470 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded5561700 1 --2- 172.21.15.123:0/1973367218 >> v2:172.21.15.177:3300/0 conn(0x7fded00b0420 0x7fded015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:18.443 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded5d62700 1 --2- 172.21.15.123:0/1973367218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded6563700 1 --2- 172.21.15.123:0/1973367218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded006b8f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fded006f160 tx=0x7fdec8009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da195f3507111486 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded4d60700 1 -- 172.21.15.123:0/1973367218 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec800e070 con 0x7fded006b470 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded4d60700 1 -- 172.21.15.123:0/1973367218 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec800ba40 con 0x7fded006b470 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded4d60700 1 -- 172.21.15.123:0/1973367218 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec80092e0 con 0x7fded006b470 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded7fc6700 1 -- 172.21.15.123:0/1973367218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 msgr2=0x7fded006b8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded7fc6700 1 --2- 172.21.15.123:0/1973367218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded006b8f0 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7fded006f160 tx=0x7fdec8009840 comp rx=0 tx=0).stop 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded7fc6700 1 -- 172.21.15.123:0/1973367218 shutdown_connections 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded7fc6700 1 --2- 172.21.15.123:0/1973367218 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded006b8f0 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.444 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded7fc6700 1 --2- 172.21.15.123:0/1973367218 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c8680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded7fc6700 1 --2- 172.21.15.123:0/1973367218 >> v2:172.21.15.177:3300/0 conn(0x7fded00b0420 0x7fded015a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.438+0000 7fded7fc6700 1 -- 172.21.15.123:0/1973367218 >> 172.21.15.123:0/1973367218 conn(0x7fded0068130 msgr2=0x7fded00ce0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.439+0000 7fded7fc6700 1 -- 172.21.15.123:0/1973367218 shutdown_connections 2023-10-06T04:33:18.445 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.439+0000 7fded7fc6700 1 -- 172.21.15.123:0/1973367218 wait complete. 2023-10-06T04:33:18.447 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.440+0000 7fded7fc6700 1 Processor -- start 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.440+0000 7fded7fc6700 1 -- start start 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded7fc6700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded00c3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded7fc6700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c2500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded7fc6700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fded00b0420 0x7fded014a130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded7fc6700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c34f0 con 0x7fded006b470 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded7fc6700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c5590 con 0x7fded00b0420 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded7fc6700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fded00c2f30 con 0x7fded00b0050 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded00c3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded00c3dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:41994/0 (socket says 172.21.15.123:41994) 2023-10-06T04:33:18.448 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 -- 172.21.15.123:0/21374243 learned_addr learned my addr 172.21.15.123:0/21374243 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5561700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c2500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded6563700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fded00b0420 0x7fded014a130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 -- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 msgr2=0x7fded00c2500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c2500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 -- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fded00b0420 msgr2=0x7fded014a130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fded00b0420 0x7fded014a130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 -- 172.21.15.123:0/21374243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec8009ae0 con 0x7fded006b470 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.441+0000 7fded5d62700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded00c3dd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdec000e900 tx=0x7fdec0008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.442+0000 7fdec5ffb700 1 -- 172.21.15.123:0/21374243 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000eea0 con 0x7fded006b470 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.442+0000 7fdec5ffb700 1 -- 172.21.15.123:0/21374243 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec000c5e0 con 0x7fded006b470 2023-10-06T04:33:18.449 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.442+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fded00c3210 con 0x7fded006b470 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.442+0000 7fdec5ffb700 1 -- 172.21.15.123:0/21374243 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec000c980 con 0x7fded006b470 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.442+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fded00c4310 con 0x7fded006b470 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.443+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fded00c5870 con 0x7fded006b470 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.443+0000 7fdec5ffb700 1 -- 172.21.15.123:0/21374243 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fdec000cd30 con 0x7fded006b470 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.443+0000 7fdec5ffb700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdeb4066f30 0x7fdeb40693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.443+0000 7fdec5ffb700 1 -- 172.21.15.123:0/21374243 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fdec0084240 con 0x7fded006b470 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.443+0000 7fded5561700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdeb4066f30 0x7fdeb40693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.450 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.444+0000 7fded5561700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdeb4066f30 0x7fdeb40693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fded00c1640 tx=0x7fdecc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.451 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.447+0000 7fdec5ffb700 1 -- 172.21.15.123:0/21374243 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fdec0055080 con 0x7fded006b470 2023-10-06T04:33:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.630+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7fded00c56d0 con 0x7fdeb4066f30 2023-10-06T04:33:18.633 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.630+0000 7fdec5ffb700 1 -- 172.21.15.123:0/21374243 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fdecc00f6e0 con 0x7fdeb4066f30 2023-10-06T04:33:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdeb4066f30 msgr2=0x7fdeb40693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdeb4066f30 0x7fdeb40693e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fded00c1640 tx=0x7fdecc008040 comp rx=0 tx=0).stop 2023-10-06T04:33:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 msgr2=0x7fded00c3dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded00c3dd0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7fdec000e900 tx=0x7fdec0008040 comp rx=0 tx=0).stop 2023-10-06T04:33:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 shutdown_connections 2023-10-06T04:33:18.635 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fdeb4066f30 0x7fdeb40693e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fded006b470 0x7fded00c3dd0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fded00b0420 0x7fded014a130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 --2- 172.21.15.123:0/21374243 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fded00b0050 0x7fded00c2500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 >> 172.21.15.123:0/21374243 conn(0x7fded0068130 msgr2=0x7fded015c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 shutdown_connections 2023-10-06T04:33:18.636 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.632+0000 7fded7fc6700 1 -- 172.21.15.123:0/21374243 wait complete. 2023-10-06T04:33:18.636 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 3 to deep-scrub 2023-10-06T04:33:18.646 INFO:tasks.ceph:Scrubbing osd.4 2023-10-06T04:33:18.647 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:18.767 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.765+0000 7f0450085700 1 Processor -- start 2023-10-06T04:33:18.768 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.765+0000 7f0450085700 1 -- start start 2023-10-06T04:33:18.770 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f0450085700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f04480af3a0 0x7f0448158e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.770 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f0450085700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0448067c80 0x7f0448159370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.770 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f0450085700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f044806add0 con 0x7f04480678b0 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f0450085700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04481598b0 con 0x7f0448067c80 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f0450085700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f044806c270 con 0x7f04480af3a0 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044e622700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0448067c80 0x7f0448159370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044e622700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0448067c80 0x7f0448159370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42000/0 (socket says 172.21.15.123:42000) 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044e622700 1 -- 172.21.15.123:0/2506453246 learned_addr learned my addr 172.21.15.123:0/2506453246 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044d620700 1 --2- 172.21.15.123:0/2506453246 >> v2:172.21.15.177:3300/0 conn(0x7f04480af3a0 0x7f0448158e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044e622700 1 -- 172.21.15.123:0/2506453246 >> v2:172.21.15.177:3300/0 conn(0x7f04480af3a0 msgr2=0x7f0448158e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044e622700 1 --2- 172.21.15.123:0/2506453246 >> v2:172.21.15.177:3300/0 conn(0x7f04480af3a0 0x7f0448158e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044e622700 1 -- 172.21.15.123:0/2506453246 >> v1:172.21.15.177:6789/0 conn(0x7f04480678b0 legacy=0x7f04480aec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:18.771 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.766+0000 7f044e622700 1 -- 172.21.15.123:0/2506453246 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0448068570 con 0x7f0448067c80 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f044d620700 1 --2- 172.21.15.123:0/2506453246 >> v2:172.21.15.177:3300/0 conn(0x7f04480af3a0 0x7f0448158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f044e622700 1 --2- 172.21.15.123:0/2506453246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f0448159370 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f044806de90 tx=0x7f0440009840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcf48f52040bd6d6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f044ce1f700 1 -- 172.21.15.123:0/2506453246 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f044000e070 con 0x7f0448067c80 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f044ce1f700 1 -- 172.21.15.123:0/2506453246 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f044000ba10 con 0x7f0448067c80 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f044ce1f700 1 -- 172.21.15.123:0/2506453246 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04400092e0 con 0x7f0448067c80 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 -- 172.21.15.123:0/2506453246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 msgr2=0x7f0448159370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 --2- 172.21.15.123:0/2506453246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f0448159370 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f044806de90 tx=0x7f0440009840 comp rx=0 tx=0).stop 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 -- 172.21.15.123:0/2506453246 shutdown_connections 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 --2- 172.21.15.123:0/2506453246 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f0448159370 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 --2- 172.21.15.123:0/2506453246 >> v2:172.21.15.177:3300/0 conn(0x7f04480af3a0 0x7f0448158e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 -- 172.21.15.123:0/2506453246 >> 172.21.15.123:0/2506453246 conn(0x7f04480cdb10 msgr2=0x7f04480abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.772 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 -- 172.21.15.123:0/2506453246 shutdown_connections 2023-10-06T04:33:18.773 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.767+0000 7f0450085700 1 -- 172.21.15.123:0/2506453246 wait complete. 2023-10-06T04:33:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.768+0000 7f0450085700 1 Processor -- start 2023-10-06T04:33:18.775 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f0450085700 1 -- start start 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f0450085700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04480678b0 0x7f044814dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f0450085700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f044815d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f0450085700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04480af3a0 0x7f044814a870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f0450085700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04480b0c00 con 0x7f0448067c80 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f0450085700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f044815e310 con 0x7f04480af3a0 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f0450085700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04480b02d0 con 0x7f04480678b0 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f044d620700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f044815d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f044d620700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f044815d8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42004/0 (socket says 172.21.15.123:42004) 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f044d620700 1 -- 172.21.15.123:0/2098059194 learned_addr learned my addr 172.21.15.123:0/2098059194 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f044de21700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04480678b0 0x7f044814dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.776 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.769+0000 7f044e622700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04480af3a0 0x7f044814a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f044d620700 1 -- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04480678b0 msgr2=0x7f044814dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f044d620700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f04480678b0 0x7f044814dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f044d620700 1 -- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04480af3a0 msgr2=0x7f044814a870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f044d620700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04480af3a0 0x7f044814a870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f044d620700 1 -- 172.21.15.123:0/2098059194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0440009ae0 con 0x7f0448067c80 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f044e622700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f04480af3a0 0x7f044814a870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f044d620700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f044815d8c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f044400c850 tx=0x7f0444008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444011600 con 0x7f0448067c80 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0444011760 con 0x7f0448067c80 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0444011b20 con 0x7f0448067c80 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04480b05b0 con 0x7f0448067c80 2023-10-06T04:33:18.777 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.770+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04480abf30 con 0x7f0448067c80 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.771+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f044806d650 con 0x7f0448067c80 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.771+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0444018350 con 0x7f0448067c80 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f043dffb700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f042c066f30 0x7f042c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f0444084460 con 0x7f0448067c80 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f044de21700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f042c066f30 0x7f042c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f043dffb700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f042c06d440 0x7f042c06f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f042c06ff00 con 0x7f042c06d440 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f04440869c0 con 0x7f0448067c80 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f044e622700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f042c06d440 0x7f042c06f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f044de21700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f042c066f30 0x7f042c0693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0438003a30 tx=0x7f0438007ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.778 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.772+0000 7f044e622700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f042c06d440 0x7f042c06f850 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.779 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.773+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== osd.4 v2:172.21.15.177:6808/119946 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f042c06ff00 con 0x7f042c06d440 2023-10-06T04:33:18.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.786+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 --> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f04480af930 con 0x7f042c06d440 2023-10-06T04:33:18.789 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.787+0000 7f043dffb700 1 -- 172.21.15.123:0/2098059194 <== osd.4 v2:172.21.15.177:6808/119946 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f04480af930 con 0x7f042c06d440 2023-10-06T04:33:18.790 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:18.790 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:18.790 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f042c06d440 msgr2=0x7f042c06f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.177:6808/119946,v1:172.21.15.177:6809/119946] conn(0x7f042c06d440 0x7f042c06f850 crc :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f042c066f30 msgr2=0x7f042c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f042c066f30 0x7f042c0693e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0438003a30 tx=0x7f0438007ea0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 msgr2=0x7f044815d8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 --2- 172.21.15.123:0/2098059194 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0448067c80 0x7f044815d8c0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f044400c850 tx=0x7f0444008040 comp rx=0 tx=0).stop 2023-10-06T04:33:18.791 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f044e622700 1 -- 172.21.15.123:0/2098059194 reap_dead start 2023-10-06T04:33:18.792 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 shutdown_connections 2023-10-06T04:33:18.792 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 >> 172.21.15.123:0/2098059194 conn(0x7f04480cdb10 msgr2=0x7f0448155210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.792 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 shutdown_connections 2023-10-06T04:33:18.792 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.788+0000 7f0450085700 1 -- 172.21.15.123:0/2098059194 wait complete. 2023-10-06T04:33:18.802 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-10-06T04:33:18.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.917+0000 7f0ae4727700 1 Processor -- start 2023-10-06T04:33:18.920 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.917+0000 7f0ae4727700 1 -- start start 2023-10-06T04:33:18.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.918+0000 7f0ae4727700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0adc159750 0x7f0adc06ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.918+0000 7f0ae4727700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0adc06b390 con 0x7f0adc159380 2023-10-06T04:33:18.923 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.918+0000 7f0ae4727700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0adc06c1d0 con 0x7f0adc068d80 2023-10-06T04:33:18.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.918+0000 7f0ae4727700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0adc06b5c0 con 0x7f0adc159750 2023-10-06T04:33:18.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae2cc4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0adc159750 0x7f0adc06ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.924 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae24c3700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f0adc159380 0x7f0adc068670 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55452/0 (socket says 172.21.15.123:55452) 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae2cc4700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f0adc159750 0x7f0adc06ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42020/0 (socket says 172.21.15.123:42020) 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae24c3700 1 -- 172.21.15.123:0/714904672 learned_addr learned my addr 172.21.15.123:0/714904672 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae2cc4700 1 -- 172.21.15.123:0/714904672 >> v1:172.21.15.177:6789/0 conn(0x7f0adc068d80 legacy=0x7f0adc0c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae2cc4700 1 -- 172.21.15.123:0/714904672 >> v1:172.21.15.123:6789/0 conn(0x7f0adc159380 legacy=0x7f0adc068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae2cc4700 1 -- 172.21.15.123:0/714904672 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0adc06b8a0 con 0x7f0adc159750 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae14c1700 1 -- 172.21.15.123:0/714904672 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1564866216 0 0) 0x7f0adc06b390 con 0x7f0adc159380 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae2cc4700 1 --2- 172.21.15.123:0/714904672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc06ae50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009e60 tx=0x7f0ad4009780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92eae258a9bb1953 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae14c1700 1 -- 172.21.15.123:0/714904672 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad400e070 con 0x7f0adc159750 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae14c1700 1 -- 172.21.15.123:0/714904672 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad400bbf0 con 0x7f0adc159750 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.919+0000 7f0ae14c1700 1 -- 172.21.15.123:0/714904672 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad4012430 con 0x7f0adc159750 2023-10-06T04:33:18.925 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.920+0000 7f0ae4727700 1 -- 172.21.15.123:0/714904672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 msgr2=0x7f0adc06ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.920+0000 7f0ae4727700 1 --2- 172.21.15.123:0/714904672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc06ae50 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f0ad4009e60 tx=0x7f0ad4009780 comp rx=0 tx=0).stop 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.920+0000 7f0ae4727700 1 -- 172.21.15.123:0/714904672 shutdown_connections 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.920+0000 7f0ae4727700 1 --2- 172.21.15.123:0/714904672 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc06ae50 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.920+0000 7f0ae4727700 1 -- 172.21.15.123:0/714904672 >> 172.21.15.123:0/714904672 conn(0x7f0adc0c9d00 msgr2=0x7f0adc06da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.920+0000 7f0ae4727700 1 -- 172.21.15.123:0/714904672 shutdown_connections 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.920+0000 7f0ae4727700 1 -- 172.21.15.123:0/714904672 wait complete. 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.921+0000 7f0ae4727700 1 Processor -- start 2023-10-06T04:33:18.926 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.921+0000 7f0ae4727700 1 -- start start 2023-10-06T04:33:18.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0adc068d80 0x7f0adc14a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc159380 0x7f0adc0a5c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc0a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc14a850 con 0x7f0adc159750 2023-10-06T04:33:18.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc0c52b0 con 0x7f0adc159380 2023-10-06T04:33:18.929 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0adc16a760 con 0x7f0adc068d80 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc0a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc0a61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42028/0 (socket says 172.21.15.123:42028) 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 -- 172.21.15.123:0/1883640526 learned_addr learned my addr 172.21.15.123:0/1883640526 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 -- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0adc068d80 msgr2=0x7f0adc14a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae24c3700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0adc068d80 0x7f0adc14a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0adc068d80 0x7f0adc14a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae1cc2700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc159380 0x7f0adc0a5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 -- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc159380 msgr2=0x7f0adc0a5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc159380 0x7f0adc0a5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 -- 172.21.15.123:0/1883640526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ad4009a20 con 0x7f0adc159750 2023-10-06T04:33:18.930 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae24c3700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0adc068d80 0x7f0adc14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae1cc2700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc159380 0x7f0adc0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae2cc4700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc0a61d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400b6c0 tx=0x7f0ad4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ad27fc700 1 -- 172.21.15.123:0/1883640526 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad401e710 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ad27fc700 1 -- 172.21.15.123:0/1883640526 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ad401e870 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0adc0c4980 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ad27fc700 1 -- 172.21.15.123:0/1883640526 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ad401ec30 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.922+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0adc0c4f40 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.923+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0adc06c9b0 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.924+0000 7f0ad27fc700 1 -- 172.21.15.123:0/1883640526 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f0ad4024070 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.924+0000 7f0ad27fc700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0ac0066f30 0x7f0ac00693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.924+0000 7f0ae24c3700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0ac0066f30 0x7f0ac00693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.924+0000 7f0ad27fc700 1 -- 172.21.15.123:0/1883640526 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f0ad4086f70 con 0x7f0adc159750 2023-10-06T04:33:18.931 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.925+0000 7f0ae24c3700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0ac0066f30 0x7f0ac00693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0018d0 tx=0x7f0acc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:18.932 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:18.929+0000 7f0ad27fc700 1 -- 172.21.15.123:0/1883640526 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f0ad40543d0 con 0x7f0adc159750 2023-10-06T04:33:19.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.109+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7f0adc0c53f0 con 0x7f0ac0066f30 2023-10-06T04:33:19.111 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.109+0000 7f0ad27fc700 1 -- 172.21.15.123:0/1883640526 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f0adc0c53f0 con 0x7f0ac0066f30 2023-10-06T04:33:19.113 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0ac0066f30 msgr2=0x7f0ac00693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0ac0066f30 0x7f0ac00693e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f0acc0018d0 tx=0x7f0acc009040 comp rx=0 tx=0).stop 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 msgr2=0x7f0adc0a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc0a61d0 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f0ad400b6c0 tx=0x7f0ad4018000 comp rx=0 tx=0).stop 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 shutdown_connections 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f0ac0066f30 0x7f0ac00693e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f0adc159750 0x7f0adc0a61d0 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f0adc068d80 0x7f0adc14a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 --2- 172.21.15.123:0/1883640526 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f0adc159380 0x7f0adc0a5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.114 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 >> 172.21.15.123:0/1883640526 conn(0x7f0adc0c9d00 msgr2=0x7f0adc0aea10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.115 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 shutdown_connections 2023-10-06T04:33:19.115 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.111+0000 7f0ae4727700 1 -- 172.21.15.123:0/1883640526 wait complete. 2023-10-06T04:33:19.115 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 4 to deep-scrub 2023-10-06T04:33:19.125 INFO:tasks.ceph:Scrubbing osd.5 2023-10-06T04:33:19.125 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:19.246 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.244+0000 7ff392253700 1 Processor -- start 2023-10-06T04:33:19.247 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.244+0000 7ff392253700 1 -- start start 2023-10-06T04:33:19.250 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff392253700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 0x7ff38c0ac850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff392253700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7ff38c0acd90 0x7ff38c0adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff392253700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff38c0ae960 con 0x7ff38c0ae500 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff392253700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff38c157ba0 con 0x7ff38c0acd90 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff392253700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0ad350 con 0x7ff38c1573e0 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38bfff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7ff38c0ae500 0x7ff38c157ff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55468/0 (socket says 172.21.15.123:55468) 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38bfff700 1 -- 172.21.15.123:0/986275047 learned_addr learned my addr 172.21.15.123:0/986275047 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 0x7ff38c0ac850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.251 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1285672272 0 0) 0x7ff384001490 con 0x7ff38c0ae500 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff374003570 con 0x7ff38c0ae500 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38affd700 1 --2- 172.21.15.123:0/986275047 >> v2:172.21.15.177:3300/0 conn(0x7ff38c0acd90 0x7ff38c0adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2263829401 0 0) 0x7ff374003570 con 0x7ff38c0ae500 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 msgr2=0x7ff38c0ac850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 0x7ff38c0ac850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 >> v2:172.21.15.177:3300/0 conn(0x7ff38c0acd90 msgr2=0x7ff38c0adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 --2- 172.21.15.123:0/986275047 >> v2:172.21.15.177:3300/0 conn(0x7ff38c0acd90 0x7ff38c0adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0ad630 con 0x7ff38c0ae500 2023-10-06T04:33:19.252 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7ff384001980 con 0x7ff38c0ae500 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38affd700 1 --2- 172.21.15.123:0/986275047 >> v2:172.21.15.177:3300/0 conn(0x7ff38c0acd90 0x7ff38c0adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.245+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 >> v1:172.21.15.123:6789/0 conn(0x7ff38c0ae500 legacy=0x7ff38c157ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff374003a60 0x7ff374003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff374004810 0x7ff374004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 0x7ff3740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c157ba0 con 0x7ff374003a60 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0ad350 con 0x7ff374004810 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff37400bae0 con 0x7ff38c1573e0 2023-10-06T04:33:19.253 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff374003a60 0x7ff374003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38affd700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff374004810 0x7ff374004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38bfff700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 0x7ff3740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38b7fe700 1 -- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 msgr2=0x7ff3740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff38c1573e0 0x7ff3740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38b7fe700 1 -- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff374004810 msgr2=0x7ff374004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff374004810 0x7ff374004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38b7fe700 1 -- 172.21.15.123:0/986275047 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff374003570 con 0x7ff374003a60 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff374003a60 0x7ff374003e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff380009c00 tx=0x7ff3800091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07fafd7fbe770bc server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.254 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000e710 con 0x7ff374003a60 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff38000e870 con 0x7ff374003a60 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.246+0000 7ff38a7fc700 1 -- 172.21.15.123:0/986275047 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff38000ec30 con 0x7ff374003a60 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.247+0000 7ff392253700 1 -- 172.21.15.123:0/986275047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff374003a60 msgr2=0x7ff374003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.247+0000 7ff392253700 1 --2- 172.21.15.123:0/986275047 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff374003a60 0x7ff374003e70 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7ff380009c00 tx=0x7ff3800091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.247+0000 7ff38bfff700 1 -- 172.21.15.123:0/986275047 reap_dead start 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.247+0000 7ff392253700 1 -- 172.21.15.123:0/986275047 shutdown_connections 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.247+0000 7ff392253700 1 -- 172.21.15.123:0/986275047 >> 172.21.15.123:0/986275047 conn(0x7ff38c0cdb10 msgr2=0x7ff38c0abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.247+0000 7ff392253700 1 -- 172.21.15.123:0/986275047 shutdown_connections 2023-10-06T04:33:19.255 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.247+0000 7ff392253700 1 -- 172.21.15.123:0/986275047 wait complete. 2023-10-06T04:33:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.248+0000 7ff392253700 1 Processor -- start 2023-10-06T04:33:19.256 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.248+0000 7ff392253700 1 -- start start 2023-10-06T04:33:19.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff392253700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff374003a60 0x7ff38c14f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff392253700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff38c0ca830 0x7ff38c0c9060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff392253700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c14ccc0 0x7ff38c0cb2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff392253700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c14edd0 con 0x7ff38c0ca830 2023-10-06T04:33:19.259 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff392253700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0c95a0 con 0x7ff38c14ccc0 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff392253700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff38c0ca3a0 con 0x7ff374003a60 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff38c0ca830 0x7ff38c0c9060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff38c0ca830 0x7ff38c0c9060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42046/0 (socket says 172.21.15.123:42046) 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 -- 172.21.15.123:0/2850305030 learned_addr learned my addr 172.21.15.123:0/2850305030 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 -- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff374003a60 msgr2=0x7ff38c14f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff374003a60 0x7ff38c14f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7ff374003a60 0x7ff38c14f8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38bfff700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c14ccc0 0x7ff38c0cb2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.260 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 -- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c14ccc0 msgr2=0x7ff38c0cb2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7ff38c14ccc0 0x7ff38c0cb2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 -- 172.21.15.123:0/2850305030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff38c0c9790 con 0x7ff38c0ca830 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.249+0000 7ff38affd700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff38c0ca830 0x7ff38c0c9060 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff37c011990 tx=0x7ff37c008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.250+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00c070 con 0x7ff38c0ca830 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.250+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff37c011ed0 con 0x7ff38c0ca830 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.250+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff38c0c9a70 con 0x7ff38c0ca830 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.250+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff37c00fb30 con 0x7ff38c0ca830 2023-10-06T04:33:19.261 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.250+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff38c0ca0b0 con 0x7ff38c0ca830 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.250+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7ff38c06d650 con 0x7ff38c0ca830 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.251+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7ff37c027480 con 0x7ff38c0ca830 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.251+0000 7ff3737fe700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff378066f30 0x7ff3780693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7ff37c084a70 con 0x7ff38c0ca830 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff378066f30 0x7ff3780693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff3737fe700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7ff37806d440 0x7ff37806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7ff37806ff00 con 0x7ff37806d440 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff37c084d60 con 0x7ff38c0ca830 2023-10-06T04:33:19.262 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff38bfff700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7ff37806d440 0x7ff37806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff38b7fe700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff378066f30 0x7ff3780693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff380009c00 tx=0x7ff38001a000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.252+0000 7ff38bfff700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7ff37806d440 0x7ff37806f850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.263 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.253+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== osd.5 v2:172.21.15.177:6800/119947 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7ff37806ff00 con 0x7ff37806d440 2023-10-06T04:33:19.269 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.267+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 --> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7ff38c06f420 con 0x7ff37806d440 2023-10-06T04:33:19.270 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff3737fe700 1 -- 172.21.15.123:0/2850305030 <== osd.5 v2:172.21.15.177:6800/119947 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7ff38c06f420 con 0x7ff37806d440 2023-10-06T04:33:19.270 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:19.270 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:19.271 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:19.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7ff37806d440 msgr2=0x7ff37806f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.177:6800/119947,v1:172.21.15.177:6801/119947] conn(0x7ff37806d440 0x7ff37806f850 crc :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff378066f30 msgr2=0x7ff3780693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7ff378066f30 0x7ff3780693e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ff380009c00 tx=0x7ff38001a000 comp rx=0 tx=0).stop 2023-10-06T04:33:19.272 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff38c0ca830 msgr2=0x7ff38c0c9060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 --2- 172.21.15.123:0/2850305030 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7ff38c0ca830 0x7ff38c0c9060 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7ff37c011990 tx=0x7ff37c008040 comp rx=0 tx=0).stop 2023-10-06T04:33:19.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff38bfff700 1 -- 172.21.15.123:0/2850305030 reap_dead start 2023-10-06T04:33:19.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 shutdown_connections 2023-10-06T04:33:19.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 >> 172.21.15.123:0/2850305030 conn(0x7ff38c0cdb10 msgr2=0x7ff38c0b21a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 shutdown_connections 2023-10-06T04:33:19.273 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.268+0000 7ff392253700 1 -- 172.21.15.123:0/2850305030 wait complete. 2023-10-06T04:33:19.282 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-10-06T04:33:19.399 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.397+0000 7fd22fc98700 1 Processor -- start 2023-10-06T04:33:19.400 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.397+0000 7fd22fc98700 1 -- start start 2023-10-06T04:33:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22fc98700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd228157140 0x7fd22806bf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22fc98700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fd228157510 0x7fd2280aba30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22fc98700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22fc98700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd22806aa70 con 0x7fd228157140 2023-10-06T04:33:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22fc98700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2280cb810 con 0x7fd228157510 2023-10-06T04:33:19.402 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22fc98700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280abf70 con 0x7fd2280c3840 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22da34700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd228157140 0x7fd22806bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22da34700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fd228157140 0x7fd22806bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42054/0 (socket says 172.21.15.123:42054) 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.398+0000 7fd22da34700 1 -- 172.21.15.123:0/2621170340 learned_addr learned my addr 172.21.15.123:0/2621170340 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22e235700 1 --2- 172.21.15.123:0/2621170340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22da34700 1 -- 172.21.15.123:0/2621170340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2280c3840 msgr2=0x7fd2280c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22d233700 1 --2- 172.21.15.123:0/2621170340 >> v2:172.21.15.177:3300/0 conn(0x7fd228157510 0x7fd2280aba30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22da34700 1 --2- 172.21.15.123:0/2621170340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22da34700 1 -- 172.21.15.123:0/2621170340 >> v2:172.21.15.177:3300/0 conn(0x7fd228157510 msgr2=0x7fd2280aba30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22da34700 1 --2- 172.21.15.123:0/2621170340 >> v2:172.21.15.177:3300/0 conn(0x7fd228157510 0x7fd2280aba30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.403 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22da34700 1 -- 172.21.15.123:0/2621170340 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22806a4b0 con 0x7fd228157140 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22da34700 1 --2- 172.21.15.123:0/2621170340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd22806bf70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd22815ad90 tx=0x7fd218009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db1f4855d9125c1d server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22ca32700 1 -- 172.21.15.123:0/2621170340 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800b860 con 0x7fd228157140 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22ca32700 1 -- 172.21.15.123:0/2621170340 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd21800b9c0 con 0x7fd228157140 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22ca32700 1 -- 172.21.15.123:0/2621170340 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd21800bd60 con 0x7fd228157140 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22fc98700 1 -- 172.21.15.123:0/2621170340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 msgr2=0x7fd22806bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22fc98700 1 --2- 172.21.15.123:0/2621170340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd22806bf70 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fd22815ad90 tx=0x7fd218009250 comp rx=0 tx=0).stop 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22fc98700 1 -- 172.21.15.123:0/2621170340 shutdown_connections 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22fc98700 1 --2- 172.21.15.123:0/2621170340 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22fc98700 1 --2- 172.21.15.123:0/2621170340 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd22806bf70 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.404 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22fc98700 1 --2- 172.21.15.123:0/2621170340 >> v2:172.21.15.177:3300/0 conn(0x7fd228157510 0x7fd2280aba30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.399+0000 7fd22fc98700 1 -- 172.21.15.123:0/2621170340 >> 172.21.15.123:0/2621170340 conn(0x7fd2280c9d00 msgr2=0x7fd22806da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.400+0000 7fd22fc98700 1 -- 172.21.15.123:0/2621170340 shutdown_connections 2023-10-06T04:33:19.405 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.400+0000 7fd22fc98700 1 -- 172.21.15.123:0/2621170340 wait complete. 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 Processor -- start 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 -- start start 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd2280c7d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd228157510 0x7fd22814f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280c8290 con 0x7fd228157140 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2280c4f10 con 0x7fd2280c3840 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.401+0000 7fd22fc98700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd22814ff20 con 0x7fd228157510 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd2280c7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.408 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd2280c7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42070/0 (socket says 172.21.15.123:42070) 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 -- 172.21.15.123:0/3806531046 learned_addr learned my addr 172.21.15.123:0/3806531046 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22da34700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22e235700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd228157510 0x7fd22814f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 -- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd228157510 msgr2=0x7fd22814f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd228157510 0x7fd22814f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 -- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2280c3840 msgr2=0x7fd2280c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 -- 172.21.15.123:0/3806531046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2180094f0 con 0x7fd228157140 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22d233700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd2280c7d50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd22400c8b0 tx=0x7fd224008040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.409 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd21dffb700 1 -- 172.21.15.123:0/3806531046 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224011600 con 0x7fd228157140 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd21dffb700 1 -- 172.21.15.123:0/3806531046 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd224011780 con 0x7fd228157140 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd2280c4300 con 0x7fd228157140 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd21dffb700 1 -- 172.21.15.123:0/3806531046 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd224011b40 con 0x7fd228157140 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.402+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd2280c48c0 con 0x7fd228157140 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.403+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd22806c9b0 con 0x7fd228157140 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.404+0000 7fd21dffb700 1 -- 172.21.15.123:0/3806531046 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fd224021350 con 0x7fd228157140 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.404+0000 7fd21dffb700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd20c066f30 0x7fd20c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.410 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.404+0000 7fd21dffb700 1 -- 172.21.15.123:0/3806531046 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fd224084020 con 0x7fd228157140 2023-10-06T04:33:19.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.404+0000 7fd22da34700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd20c066f30 0x7fd20c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.404+0000 7fd22da34700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd20c066f30 0x7fd20c0693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd22815ad90 tx=0x7fd218017000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.411 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.408+0000 7fd21dffb700 1 -- 172.21.15.123:0/3806531046 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fd224054e60 con 0x7fd228157140 2023-10-06T04:33:19.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.592+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7fd22806a790 con 0x7fd20c066f30 2023-10-06T04:33:19.595 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.592+0000 7fd21dffb700 1 -- 172.21.15.123:0/3806531046 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd22806a790 con 0x7fd20c066f30 2023-10-06T04:33:19.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd20c066f30 msgr2=0x7fd20c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd20c066f30 0x7fd20c0693e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fd22815ad90 tx=0x7fd218017000 comp rx=0 tx=0).stop 2023-10-06T04:33:19.597 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 msgr2=0x7fd2280c7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd2280c7d50 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fd22400c8b0 tx=0x7fd224008040 comp rx=0 tx=0).stop 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 shutdown_connections 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fd20c066f30 0x7fd20c0693e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fd228157510 0x7fd22814f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fd2280c3840 0x7fd2280c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 --2- 172.21.15.123:0/3806531046 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fd228157140 0x7fd2280c7d50 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 >> 172.21.15.123:0/3806531046 conn(0x7fd2280c9d00 msgr2=0x7fd228156400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 shutdown_connections 2023-10-06T04:33:19.598 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.595+0000 7fd22fc98700 1 -- 172.21.15.123:0/3806531046 wait complete. 2023-10-06T04:33:19.599 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 5 to deep-scrub 2023-10-06T04:33:19.609 INFO:tasks.ceph:Scrubbing osd.6 2023-10-06T04:33:19.609 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:19.729 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.727+0000 7efe0d741700 1 Processor -- start 2023-10-06T04:33:19.730 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.727+0000 7efe0d741700 1 -- start start 2023-10-06T04:33:19.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe0d741700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe0815a570 0x7efe080d01e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe0d741700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0815ad80 con 0x7efe080d0720 2023-10-06T04:33:19.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe0d741700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe080ce630 con 0x7efe08157020 2023-10-06T04:33:19.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe0d741700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe080ce910 con 0x7efe0815a570 2023-10-06T04:33:19.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe067fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe0815a570 0x7efe080d01e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe077fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7efe080d0720 0x7efe080ceda0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55474/0 (socket says 172.21.15.123:55474) 2023-10-06T04:33:19.732 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe067fc700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7efe0815a570 0x7efe080d01e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42082/0 (socket says 172.21.15.123:42082) 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe077fe700 1 -- 172.21.15.123:0/798790327 learned_addr learned my addr 172.21.15.123:0/798790327 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe067fc700 1 -- 172.21.15.123:0/798790327 >> v1:172.21.15.177:6789/0 conn(0x7efe08157020 legacy=0x7efe080ccb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe067fc700 1 -- 172.21.15.123:0/798790327 >> v1:172.21.15.123:6789/0 conn(0x7efe080d0720 legacy=0x7efe080ceda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe067fc700 1 -- 172.21.15.123:0/798790327 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe081573f0 con 0x7efe0815a570 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.728+0000 7efe05ffb700 1 -- 172.21.15.123:0/798790327 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3605055212 0 0) 0x7efe0815ad80 con 0x7efe080d0720 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe067fc700 1 --2- 172.21.15.123:0/798790327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe0815a570 0x7efe080d01e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efdf80098e0 tx=0x7efdf80091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4fa9a310d5b7296 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe05ffb700 1 -- 172.21.15.123:0/798790327 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8009e80 con 0x7efe0815a570 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe05ffb700 1 -- 172.21.15.123:0/798790327 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf800b820 con 0x7efe0815a570 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe05ffb700 1 -- 172.21.15.123:0/798790327 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf800bba0 con 0x7efe0815a570 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe0d741700 1 -- 172.21.15.123:0/798790327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe0815a570 msgr2=0x7efe080d01e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe0d741700 1 --2- 172.21.15.123:0/798790327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe0815a570 0x7efe080d01e0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7efdf80098e0 tx=0x7efdf80091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.733 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe0d741700 1 -- 172.21.15.123:0/798790327 shutdown_connections 2023-10-06T04:33:19.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe0d741700 1 --2- 172.21.15.123:0/798790327 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe0815a570 0x7efe080d01e0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe0d741700 1 -- 172.21.15.123:0/798790327 >> 172.21.15.123:0/798790327 conn(0x7efe08068160 msgr2=0x7efe08068560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe0d741700 1 -- 172.21.15.123:0/798790327 shutdown_connections 2023-10-06T04:33:19.734 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.729+0000 7efe0d741700 1 -- 172.21.15.123:0/798790327 wait complete. 2023-10-06T04:33:19.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 Processor -- start 2023-10-06T04:33:19.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 -- start start 2023-10-06T04:33:19.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080d0720 0x7efe080c7d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe08157020 0x7efe080c7560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe0815a570 0x7efe080b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.737 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe080b47d0 con 0x7efe080d0720 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe080c8400 con 0x7efe0815a570 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe0d741700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe080b3ea0 con 0x7efe08157020 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080d0720 0x7efe080c7d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe06ffd700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080d0720 0x7efe080c7d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42092/0 (socket says 172.21.15.123:42092) 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.731+0000 7efe06ffd700 1 -- 172.21.15.123:0/3957936334 learned_addr learned my addr 172.21.15.123:0/3957936334 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe077fe700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe0815a570 0x7efe080b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe067fc700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe08157020 0x7efe080c7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe06ffd700 1 -- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe08157020 msgr2=0x7efe080c7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.738 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe06ffd700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe08157020 0x7efe080c7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe06ffd700 1 -- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe0815a570 msgr2=0x7efe080b4c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe06ffd700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe0815a570 0x7efe080b4c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe06ffd700 1 -- 172.21.15.123:0/3957936334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf8009420 con 0x7efe080d0720 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe077fe700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7efe0815a570 0x7efe080b4c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe067fc700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7efe08157020 0x7efe080c7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe06ffd700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080d0720 0x7efe080c7d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b600 tx=0x7efdf00091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000f510 con 0x7efe080d0720 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf000f670 con 0x7efe080d0720 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe080b4180 con 0x7efe080d0720 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf000fa30 con 0x7efe080d0720 2023-10-06T04:33:19.739 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.732+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0814b1f0 con 0x7efe080d0720 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.733+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7efe08088610 con 0x7efe080d0720 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.733+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7efdf001a400 con 0x7efe080d0720 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efdfeffd700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efde8066f30 0x7efde80693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7efdf0083fc0 con 0x7efe080d0720 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efe067fc700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efde8066f30 0x7efde80693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efdfeffd700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7efde806d440 0x7efde806f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7efde806ff00 con 0x7efde806d440 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7efdf0086520 con 0x7efe080d0720 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efe067fc700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efde8066f30 0x7efde80693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efdf80030c0 tx=0x7efdf8016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.740 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.734+0000 7efe077fe700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7efde806d440 0x7efde806f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.735+0000 7efe077fe700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7efde806d440 0x7efde806f850 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.741 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.735+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== osd.6 v2:172.21.15.177:6816/119976 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7efde806ff00 con 0x7efde806d440 2023-10-06T04:33:19.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.749+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 --> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7efe0804bb80 con 0x7efde806d440 2023-10-06T04:33:19.751 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.749+0000 7efdfeffd700 1 -- 172.21.15.123:0/3957936334 <== osd.6 v2:172.21.15.177:6816/119976 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7efe0804bb80 con 0x7efde806d440 2023-10-06T04:33:19.752 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:19.752 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:19.752 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:19.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.749+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7efde806d440 msgr2=0x7efde806f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.749+0000 7efe0d741700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.177:6816/119976,v1:172.21.15.177:6817/119976] conn(0x7efde806d440 0x7efde806f850 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efde8066f30 msgr2=0x7efde80693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7efde8066f30 0x7efde80693e0 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7efdf80030c0 tx=0x7efdf8016000 comp rx=0 tx=0).stop 2023-10-06T04:33:19.753 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080d0720 msgr2=0x7efe080c7d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 --2- 172.21.15.123:0/3957936334 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7efe080d0720 0x7efe080c7d00 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7efdf000b600 tx=0x7efdf00091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe077fe700 1 -- 172.21.15.123:0/3957936334 reap_dead start 2023-10-06T04:33:19.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 shutdown_connections 2023-10-06T04:33:19.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 >> 172.21.15.123:0/3957936334 conn(0x7efe08068160 msgr2=0x7efe080ceda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 shutdown_connections 2023-10-06T04:33:19.754 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.750+0000 7efe0d741700 1 -- 172.21.15.123:0/3957936334 wait complete. 2023-10-06T04:33:19.764 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-10-06T04:33:19.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.879+0000 7f3734fab700 1 Processor -- start 2023-10-06T04:33:19.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.879+0000 7f3734fab700 1 -- start start 2023-10-06T04:33:19.885 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.880+0000 7f3734fab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 0x7f373006ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.880+0000 7f3734fab700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373006b390 con 0x7f3730068d80 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.880+0000 7f3734fab700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f373006c1d0 con 0x7f3730159380 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.880+0000 7f3734fab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f373006b5c0 con 0x7f3730159750 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.880+0000 7f372dd9b700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f3730068d80 0x7f37300c7010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55486/0 (socket says 172.21.15.123:55486) 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.880+0000 7f372dd9b700 1 -- 172.21.15.123:0/3565929782 learned_addr learned my addr 172.21.15.123:0/3565929782 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372ed9d700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 0x7f373006ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3619041201 0 0) 0x7f373006b390 con 0x7f3730068d80 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3718003570 con 0x7f3730068d80 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2042097379 0 0) 0x7f3718003570 con 0x7f3730068d80 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 msgr2=0x7f373006ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.886 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 0x7f373006ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 >> v1:172.21.15.177:6789/0 conn(0x7f3730159380 legacy=0x7f3730068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f373006b8a0 con 0x7f3730068d80 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f3724001470 con 0x7f3730068d80 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 >> v1:172.21.15.123:6789/0 conn(0x7f3730068d80 legacy=0x7f37300c7010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 0x7f3718003a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3718004010 0x7f3718004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718004d50 0x7f37180051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f373006b5c0 con 0x7f3718004d50 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f371800bbf0 con 0x7f3718004010 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f371800dfc0 con 0x7f3730159750 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372ed9d700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718004d50 0x7f37180051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.887 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372dd9b700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3718004010 0x7f3718004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372e59c700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 0x7f3718003a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372ed9d700 1 -- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 msgr2=0x7f3718003a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372ed9d700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 0x7f3718003a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372ed9d700 1 -- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3718004010 msgr2=0x7f3718004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372ed9d700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f3718004010 0x7f3718004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372ed9d700 1 -- 172.21.15.123:0/3565929782 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3718003570 con 0x7f3718004d50 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.881+0000 7f372e59c700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3730159750 0x7f3718003a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f372ed9d700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718004d50 0x7f37180051c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f37280094d0 tx=0x7f372800b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7263ef43b4547197 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372800ea90 con 0x7f3718004d50 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372800f040 con 0x7f3718004d50 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f372d59a700 1 -- 172.21.15.123:0/3565929782 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372800ee20 con 0x7f3718004d50 2023-10-06T04:33:19.888 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f3734fab700 1 -- 172.21.15.123:0/3565929782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718004d50 msgr2=0x7f37180051c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f3734fab700 1 --2- 172.21.15.123:0/3565929782 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3718004d50 0x7f37180051c0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f37280094d0 tx=0x7f372800b040 comp rx=0 tx=0).stop 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f372ed9d700 1 -- 172.21.15.123:0/3565929782 reap_dead start 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f3734fab700 1 -- 172.21.15.123:0/3565929782 shutdown_connections 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f3734fab700 1 -- 172.21.15.123:0/3565929782 >> 172.21.15.123:0/3565929782 conn(0x7f37300c9d00 msgr2=0x7f373006da50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f3734fab700 1 -- 172.21.15.123:0/3565929782 shutdown_connections 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.882+0000 7f3734fab700 1 -- 172.21.15.123:0/3565929782 wait complete. 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.883+0000 7f3734fab700 1 Processor -- start 2023-10-06T04:33:19.889 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.883+0000 7f3734fab700 1 -- start start 2023-10-06T04:33:19.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f3734fab700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3718004d50 0x7f37300c5ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f3734fab700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3730155180 0x7f37301555f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.891 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f3734fab700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37300ab3d0 0x7f37300ab850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f3734fab700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300ac140 con 0x7f3730155180 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f3734fab700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37300acff0 con 0x7f37300ab3d0 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f3734fab700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37300a98a0 con 0x7f3718004d50 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3730155180 0x7f37301555f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372dd9b700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3730155180 0x7f37301555f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42100/0 (socket says 172.21.15.123:42100) 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372dd9b700 1 -- 172.21.15.123:0/3801927118 learned_addr learned my addr 172.21.15.123:0/3801927118 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372dd9b700 1 -- 172.21.15.123:0/3801927118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3718004d50 msgr2=0x7f37300c5ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372ed9d700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37300ab3d0 0x7f37300ab850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372dd9b700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3718004d50 0x7f37300c5ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.892 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372dd9b700 1 -- 172.21.15.123:0/3801927118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37300ab3d0 msgr2=0x7f37300ab850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.884+0000 7f372dd9b700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37300ab3d0 0x7f37300ab850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.885+0000 7f372dd9b700 1 -- 172.21.15.123:0/3801927118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37300aa3e0 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.885+0000 7f372dd9b700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3730155180 0x7f37301555f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f37240073c0 tx=0x7f3724001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.885+0000 7f371e7fc700 1 -- 172.21.15.123:0/3801927118 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3724018030 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.885+0000 7f371e7fc700 1 -- 172.21.15.123:0/3801927118 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f372400f8d0 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.885+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3730155b30 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.885+0000 7f371e7fc700 1 -- 172.21.15.123:0/3801927118 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f372400fc90 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.885+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37300aa070 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.886+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3730071710 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.886+0000 7f371e7fc700 1 -- 172.21.15.123:0/3801927118 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f372401b070 con 0x7f3730155180 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.886+0000 7f371e7fc700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3710066f30 0x7f37100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:19.893 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.887+0000 7f371e7fc700 1 -- 172.21.15.123:0/3801927118 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f3724019070 con 0x7f3730155180 2023-10-06T04:33:19.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.887+0000 7f372e59c700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3710066f30 0x7f37100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:19.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.887+0000 7f372e59c700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3710066f30 0x7f37100693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3720007a80 tx=0x7f3720009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:19.894 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:19.891+0000 7f371e7fc700 1 -- 172.21.15.123:0/3801927118 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f37240518c0 con 0x7f3730155180 2023-10-06T04:33:20.074 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.072+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7f373006bb80 con 0x7f3710066f30 2023-10-06T04:33:20.075 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.072+0000 7f371e7fc700 1 -- 172.21.15.123:0/3801927118 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f373006bb80 con 0x7f3710066f30 2023-10-06T04:33:20.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3710066f30 msgr2=0x7f37100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.077 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3710066f30 0x7f37100693e0 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f3720007a80 tx=0x7f3720009040 comp rx=0 tx=0).stop 2023-10-06T04:33:20.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3730155180 msgr2=0x7f37301555f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3730155180 0x7f37301555f0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f37240073c0 tx=0x7f3724001e50 comp rx=0 tx=0).stop 2023-10-06T04:33:20.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 shutdown_connections 2023-10-06T04:33:20.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f3710066f30 0x7f37100693e0 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.078 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f37300ab3d0 0x7f37300ab850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f3718004d50 0x7f37300c5ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 --2- 172.21.15.123:0/3801927118 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f3730155180 0x7f37301555f0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 >> 172.21.15.123:0/3801927118 conn(0x7f37300c9d00 msgr2=0x7f3730157650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:20.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 shutdown_connections 2023-10-06T04:33:20.079 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.074+0000 7f3734fab700 1 -- 172.21.15.123:0/3801927118 wait complete. 2023-10-06T04:33:20.079 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 6 to deep-scrub 2023-10-06T04:33:20.089 INFO:tasks.ceph:Scrubbing osd.7 2023-10-06T04:33:20.089 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-10-06T04:33:20.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.207+0000 7f55a3a9d700 1 Processor -- start 2023-10-06T04:33:20.210 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.208+0000 7f55a3a9d700 1 -- start start 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.208+0000 7f55a3a9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c067660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.208+0000 7f55a3a9d700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c06afd0 con 0x7f559c159620 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.208+0000 7f55a3a9d700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f559c0afe00 con 0x7f559c067ba0 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.208+0000 7f55a3a9d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c068020 con 0x7f559c159250 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.208+0000 7f55a1038700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f559c159620 0x7f559c0aed00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:55500/0 (socket says 172.21.15.123:55500) 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.208+0000 7f55a1038700 1 -- 172.21.15.123:0/2262280543 learned_addr learned my addr 172.21.15.123:0/2262280543 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a1839700 1 --2- 172.21.15.123:0/2262280543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c067660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.213 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 30166567 0 0) 0x7f559c06afd0 con 0x7f559c159620 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5588003570 con 0x7f559c159620 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 <== mon.1 v1:172.21.15.177:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2524784046 0 0) 0x7f559c0afe00 con 0x7f559c067ba0 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 --> v1:172.21.15.177:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f559c06afd0 con 0x7f559c067ba0 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a1839700 1 -- 172.21.15.123:0/2262280543 >> v1:172.21.15.177:6789/0 conn(0x7f559c067ba0 legacy=0x7f559c06b460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3647600425 0 0) 0x7f5588003570 con 0x7f559c159620 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a1839700 1 -- 172.21.15.123:0/2262280543 >> v1:172.21.15.123:6789/0 conn(0x7f559c159620 legacy=0x7f559c0aed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a1839700 1 -- 172.21.15.123:0/2262280543 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f559c06a400 con 0x7f559c159250 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a1839700 1 --2- 172.21.15.123:0/2262280543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c067660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f558c009940 tx=0x7f558c0091a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=41efef70d6e5e47 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.214 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 <== mon.2 v2:172.21.15.177:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55980015f0 con 0x7f559c159250 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 <== mon.2 v2:172.21.15.177:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f558c011420 con 0x7f559c159250 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.209+0000 7f55a0837700 1 -- 172.21.15.123:0/2262280543 <== mon.2 v2:172.21.15.177:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f558c0117a0 con 0x7f559c159250 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.210+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2262280543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 msgr2=0x7f559c067660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.210+0000 7f55a3a9d700 1 --2- 172.21.15.123:0/2262280543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c067660 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f558c009940 tx=0x7f558c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.210+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2262280543 shutdown_connections 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.210+0000 7f55a3a9d700 1 --2- 172.21.15.123:0/2262280543 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c067660 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.210+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2262280543 >> 172.21.15.123:0/2262280543 conn(0x7f559c0cdb10 msgr2=0x7f559c0abd80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.210+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2262280543 shutdown_connections 2023-10-06T04:33:20.215 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.210+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2262280543 wait complete. 2023-10-06T04:33:20.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.211+0000 7f55a3a9d700 1 Processor -- start 2023-10-06T04:33:20.219 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.211+0000 7f55a3a9d700 1 -- start start 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a3a9d700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c067ba0 0x7f559c169ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a3a9d700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c16b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a3a9d700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559c159620 0x7f559c153a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a3a9d700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c16c490 con 0x7f559c159620 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a3a9d700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f559c16b600 con 0x7f559c067ba0 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a3a9d700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f559c16b880 con 0x7f559c159250 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559c159620 0x7f559c153a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559c159620 0x7f559c153a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:42112/0 (socket says 172.21.15.123:42112) 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 -- 172.21.15.123:0/2791368966 learned_addr learned my addr 172.21.15.123:0/2791368966 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a1038700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c16b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 -- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 msgr2=0x7f559c16b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.220 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a1839700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c067ba0 0x7f559c169ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f559c159250 0x7f559c16b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 -- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c067ba0 msgr2=0x7f559c169ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f559c067ba0 0x7f559c169ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 -- 172.21.15.123:0/2791368966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f558c009420 con 0x7f559c159620 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f55a203a700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559c159620 0x7f559c153a00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f559400c860 tx=0x7f5594002700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.212+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594002030 con 0x7f559c159620 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.213+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55940021b0 con 0x7f559c159620 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.213+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f559c16bb60 con 0x7f559c159620 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.213+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5594012630 con 0x7f559c159620 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.213+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f559c16c120 con 0x7f559c159620 2023-10-06T04:33:20.221 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.213+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f559c06d650 con 0x7f559c159620 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.214+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f5594012a20 con 0x7f559c159620 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.214+0000 7f5591ffb700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5580066f30 0x7f55800693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.214+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f55940844a0 con 0x7f559c159620 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.214+0000 7f55a1839700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5580066f30 0x7f55800693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.215+0000 7f5591ffb700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f558006d440 0x7f558006f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.215+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f558006ff00 con 0x7f558006d440 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.215+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7f5594086ae0 con 0x7f559c159620 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.215+0000 7f55a1038700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f558006d440 0x7f558006f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.222 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.215+0000 7f55a1839700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5580066f30 0x7f55800693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f558c009860 tx=0x7f558c016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.215+0000 7f55a1038700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f558006d440 0x7f558006f850 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.223 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.216+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== osd.7 v2:172.21.15.177:6824/119977 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+23999 (crc 0 0 0) 0x7f558006ff00 con 0x7f558006d440 2023-10-06T04:33:20.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.229+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 --> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f559c154900 con 0x7f558006d440 2023-10-06T04:33:20.232 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.230+0000 7f5591ffb700 1 -- 172.21.15.123:0/2791368966 <== osd.7 v2:172.21.15.177:6824/119977 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f559c154900 con 0x7f558006d440 2023-10-06T04:33:20.233 INFO:teuthology.orchestra.run.smithi123.stdout:{ 2023-10-06T04:33:20.233 INFO:teuthology.orchestra.run.smithi123.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-10-06T04:33:20.233 INFO:teuthology.orchestra.run.smithi123.stdout:} 2023-10-06T04:33:20.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.230+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f558006d440 msgr2=0x7f558006f850 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.230+0000 7f55a3a9d700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.177:6824/119977,v1:172.21.15.177:6825/119977] conn(0x7f558006d440 0x7f558006f850 crc :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5580066f30 msgr2=0x7f55800693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f5580066f30 0x7f55800693e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f558c009860 tx=0x7f558c016000 comp rx=0 tx=0).stop 2023-10-06T04:33:20.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559c159620 msgr2=0x7f559c153a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 --2- 172.21.15.123:0/2791368966 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f559c159620 0x7f559c153a00 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7f559400c860 tx=0x7f5594002700 comp rx=0 tx=0).stop 2023-10-06T04:33:20.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a203a700 1 -- 172.21.15.123:0/2791368966 reap_dead start 2023-10-06T04:33:20.235 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 shutdown_connections 2023-10-06T04:33:20.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 >> 172.21.15.123:0/2791368966 conn(0x7f559c0cdb10 msgr2=0x7f559c0a9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:20.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 shutdown_connections 2023-10-06T04:33:20.236 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.231+0000 7f55a3a9d700 1 -- 172.21.15.123:0/2791368966 wait complete. 2023-10-06T04:33:20.244 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-10-06T04:33:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.359+0000 7f2187523700 1 Processor -- start 2023-10-06T04:33:20.362 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.360+0000 7f2187523700 1 -- start start 2023-10-06T04:33:20.364 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.360+0000 7f2187523700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f218015ada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2187523700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f218006aca0 0x7f2180158040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2187523700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f218015b680 con 0x7f2180158580 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2187523700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2180155270 con 0x7f218006aca0 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2187523700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218015b960 con 0x7f2180157c70 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f218006aca0 0x7f2180158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f218006aca0 0x7f2180158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36410/0 (socket says 172.21.15.123:36410) 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 -- 172.21.15.123:0/3898100727 learned_addr learned my addr 172.21.15.123:0/3898100727 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f21852bf700 1 --2- 172.21.15.123:0/3898100727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f218015ada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 -- 172.21.15.123:0/3898100727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 msgr2=0x7f218015ada0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 --2- 172.21.15.123:0/3898100727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f218015ada0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 -- 172.21.15.123:0/3898100727 >> v1:172.21.15.177:6789/0 conn(0x7f2180158580 legacy=0x7f2180068a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 -- 172.21.15.123:0/3898100727 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f218015bc40 con 0x7f218006aca0 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f2184abe700 1 --2- 172.21.15.123:0/3898100727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218006aca0 0x7f2180158040 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f21740099b0 tx=0x7f2174009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=567cf7def2f822c6 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f217ffff700 1 -- 172.21.15.123:0/3898100727 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217400b860 con 0x7f218006aca0 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f217ffff700 1 -- 172.21.15.123:0/3898100727 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f217400b9c0 con 0x7f218006aca0 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.361+0000 7f217ffff700 1 -- 172.21.15.123:0/3898100727 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f217400bd60 con 0x7f218006aca0 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 -- 172.21.15.123:0/3898100727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218006aca0 msgr2=0x7f2180158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 --2- 172.21.15.123:0/3898100727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218006aca0 0x7f2180158040 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f21740099b0 tx=0x7f2174009250 comp rx=0 tx=0).stop 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 -- 172.21.15.123:0/3898100727 shutdown_connections 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 --2- 172.21.15.123:0/3898100727 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f218015ada0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 --2- 172.21.15.123:0/3898100727 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f218006aca0 0x7f2180158040 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.366 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 -- 172.21.15.123:0/3898100727 >> 172.21.15.123:0/3898100727 conn(0x7f21800cd010 msgr2=0x7f21800cd410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 -- 172.21.15.123:0/3898100727 shutdown_connections 2023-10-06T04:33:20.367 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.362+0000 7f2187523700 1 -- 172.21.15.123:0/3898100727 wait complete. 2023-10-06T04:33:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.363+0000 7f2187523700 1 Processor -- start 2023-10-06T04:33:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.363+0000 7f2187523700 1 -- start start 2023-10-06T04:33:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2187523700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218006aca0 0x7f21800b1d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2187523700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f21800b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2187523700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2180158580 0x7f2180068c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2187523700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f218016b2b0 con 0x7f2180158580 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2187523700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21800c4330 con 0x7f218006aca0 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2187523700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f218016b500 con 0x7f2180157c70 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2180158580 0x7f2180068c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2180158580 0x7f2180068c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36422/0 (socket says 172.21.15.123:36422) 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 -- 172.21.15.123:0/1516691234 learned_addr learned my addr 172.21.15.123:0/1516691234 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2184abe700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f21800b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f21852bf700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218006aca0 0x7f21800b1d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 -- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 msgr2=0x7f21800b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f21800b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 -- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218006aca0 msgr2=0x7f21800b1d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218006aca0 0x7f21800b1d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2185ac0700 1 -- 172.21.15.123:0/1516691234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f21740094f0 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.364+0000 7f2184abe700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f21800b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.365+0000 7f2185ac0700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2180158580 0x7f2180068c50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f217000c810 tx=0x7f2170001650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.365+0000 7f217d7fa700 1 -- 172.21.15.123:0/1516691234 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170012630 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.365+0000 7f217d7fa700 1 -- 172.21.15.123:0/1516691234 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21700127b0 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.365+0000 7f217d7fa700 1 -- 172.21.15.123:0/1516691234 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2170012b70 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.365+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f218016b7e0 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.365+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f218016bda0 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.366+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f21800afe00 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.366+0000 7f217d7fa700 1 -- 172.21.15.123:0/1516691234 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2170003030 con 0x7f2180158580 2023-10-06T04:33:20.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.367+0000 7f217d7fa700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f216c066f30 0x7f216c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.367+0000 7f217d7fa700 1 -- 172.21.15.123:0/1516691234 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f2170084b40 con 0x7f2180158580 2023-10-06T04:33:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.367+0000 7f21852bf700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f216c066f30 0x7f216c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.367+0000 7f21852bf700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f216c066f30 0x7f216c0693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2178006fd0 tx=0x7f2178008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.370+0000 7f217d7fa700 1 -- 172.21.15.123:0/1516691234 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f2170051ef0 con 0x7f2180158580 2023-10-06T04:33:20.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.555+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f218006a4a0 con 0x7f216c066f30 2023-10-06T04:33:20.557 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.555+0000 7f217d7fa700 1 -- 172.21.15.123:0/1516691234 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f218006a4a0 con 0x7f216c066f30 2023-10-06T04:33:20.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f216c066f30 msgr2=0x7f216c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f216c066f30 0x7f216c0693e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f2178006fd0 tx=0x7f2178008040 comp rx=0 tx=0).stop 2023-10-06T04:33:20.559 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2180158580 msgr2=0x7f2180068c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2180158580 0x7f2180068c50 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f217000c810 tx=0x7f2170001650 comp rx=0 tx=0).stop 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 shutdown_connections 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f216c066f30 0x7f216c0693e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f2180158580 0x7f2180068c50 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f218006aca0 0x7f21800b1d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 --2- 172.21.15.123:0/1516691234 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2180157c70 0x7f21800b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 >> 172.21.15.123:0/1516691234 conn(0x7f21800cd010 msgr2=0x7f21801541a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 shutdown_connections 2023-10-06T04:33:20.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.557+0000 7f2187523700 1 -- 172.21.15.123:0/1516691234 wait complete. 2023-10-06T04:33:20.561 INFO:teuthology.orchestra.run.smithi123.stderr:instructed osd(s) 7 to deep-scrub 2023-10-06T04:33:20.571 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:33:20.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.685+0000 7fc027452700 1 Processor -- start 2023-10-06T04:33:20.688 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.686+0000 7fc027452700 1 -- start start 2023-10-06T04:33:20.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc027452700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7fc020068d80 0x7fc0200c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc027452700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0200c7c50 con 0x7fc0200c7460 2023-10-06T04:33:20.692 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc027452700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0200c7f30 con 0x7fc020155780 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc027452700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0200c8210 con 0x7fc020068d80 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc0259ef700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7fc0200c7460 0x7fc02006aa40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:41796/0 (socket says 172.21.15.123:41796) 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc0259ef700 1 -- 172.21.15.123:0/3602456617 learned_addr learned my addr 172.21.15.123:0/3602456617 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc0249ed700 1 --2- 172.21.15.123:0/3602456617 >> v2:172.21.15.177:3300/0 conn(0x7fc020068d80 0x7fc0200c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1619172827 0 0) 0x7fc0200c7c50 con 0x7fc0200c7460 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc00c003570 con 0x7fc0200c7460 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3399981830 0 0) 0x7fc00c003570 con 0x7fc0200c7460 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 >> v2:172.21.15.177:3300/0 conn(0x7fc020068d80 msgr2=0x7fc0200c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 --2- 172.21.15.123:0/3602456617 >> v2:172.21.15.177:3300/0 conn(0x7fc020068d80 0x7fc0200c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.693 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 >> v1:172.21.15.177:6789/0 conn(0x7fc020155780 legacy=0x7fc020068670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0200c84f0 con 0x7fc0200c7460 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.687+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7fc0180017f0 con 0x7fc0200c7460 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 >> v1:172.21.15.123:6789/0 conn(0x7fc0200c7460 legacy=0x7fc02006aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c003a60 0x7fc00c003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc00c004810 0x7fc00c004c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc00c0051c0 0x7fc00c00ba60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200c8210 con 0x7fc00c0051c0 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c00c0d0 con 0x7fc00c003a60 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc00c00c290 con 0x7fc00c004810 2023-10-06T04:33:20.694 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc00c0051c0 0x7fc00c00ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0251ee700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c003a60 0x7fc00c003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0249ed700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc00c004810 0x7fc00c004c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 -- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc00c004810 msgr2=0x7fc00c004c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc00c004810 0x7fc00c004c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 -- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c003a60 msgr2=0x7fc00c003e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c003a60 0x7fc00c003e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 -- 172.21.15.123:0/3602456617 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc00c003570 con 0x7fc00c0051c0 2023-10-06T04:33:20.695 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 -- 172.21.15.123:0/3602456617 reap_dead start 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc0259ef700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc00c0051c0 0x7fc00c00ba60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc01800a900 tx=0x7fc0200c7e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20866a1a2f6092c server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0180070d0 con 0x7fc00c0051c0 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc018007230 con 0x7fc00c0051c0 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.688+0000 7fc017fff700 1 -- 172.21.15.123:0/3602456617 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0180075f0 con 0x7fc00c0051c0 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.689+0000 7fc027452700 1 -- 172.21.15.123:0/3602456617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc00c0051c0 msgr2=0x7fc00c00ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.689+0000 7fc027452700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc00c0051c0 0x7fc00c00ba60 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7fc01800a900 tx=0x7fc0200c7e20 comp rx=0 tx=0).stop 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.689+0000 7fc027452700 1 -- 172.21.15.123:0/3602456617 shutdown_connections 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.689+0000 7fc027452700 1 --2- 172.21.15.123:0/3602456617 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc00c0051c0 0x7fc00c00ba60 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.696 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.689+0000 7fc027452700 1 -- 172.21.15.123:0/3602456617 >> 172.21.15.123:0/3602456617 conn(0x7fc0200c9c90 msgr2=0x7fc02006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:20.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.689+0000 7fc027452700 1 -- 172.21.15.123:0/3602456617 shutdown_connections 2023-10-06T04:33:20.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.689+0000 7fc027452700 1 -- 172.21.15.123:0/3602456617 wait complete. 2023-10-06T04:33:20.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.690+0000 7fc027452700 1 Processor -- start 2023-10-06T04:33:20.697 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.690+0000 7fc027452700 1 -- start start 2023-10-06T04:33:20.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc027452700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c0051c0 0x7fc02016aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc027452700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc020154950 0x7fc020168510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc027452700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc020168cb0 0x7fc0200c5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc027452700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc020153cd0 con 0x7fc020154950 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc027452700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0200affd0 con 0x7fc00c0051c0 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc027452700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc020168a50 con 0x7fc020168cb0 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc020154950 0x7fc020168510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc020154950 0x7fc020168510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:36426/0 (socket says 172.21.15.123:36426) 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 -- 172.21.15.123:0/1582849722 learned_addr learned my addr 172.21.15.123:0/1582849722 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:20.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0259ef700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc020168cb0 0x7fc0200c5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 -- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc020168cb0 msgr2=0x7fc0200c5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc020168cb0 0x7fc0200c5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 -- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c0051c0 msgr2=0x7fc02016aab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0251ee700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c0051c0 0x7fc02016aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c0051c0 0x7fc02016aab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0249ed700 1 -- 172.21.15.123:0/1582849722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc020154ec0 con 0x7fc020154950 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0259ef700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc020168cb0 0x7fc0200c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.691+0000 7fc0251ee700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c0051c0 0x7fc02016aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.692+0000 7fc0249ed700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc020154950 0x7fc020168510 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc01c011b10 tx=0x7fc01c0091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.692+0000 7fc0157fa700 1 -- 172.21.15.123:0/1582849722 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00f990 con 0x7fc020154950 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.692+0000 7fc0157fa700 1 -- 172.21.15.123:0/1582849722 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc01c00faf0 con 0x7fc020154950 2023-10-06T04:33:20.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.692+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0200af6a0 con 0x7fc020154950 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.692+0000 7fc0157fa700 1 -- 172.21.15.123:0/1582849722 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc01c00fe70 con 0x7fc020154950 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.692+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0200afc60 con 0x7fc020154950 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.693+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0200b0180 con 0x7fc020154950 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.693+0000 7fc0157fa700 1 -- 172.21.15.123:0/1582849722 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fc01c014650 con 0x7fc020154950 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.693+0000 7fc0157fa700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc004066f30 0x7fc0040693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.693+0000 7fc0157fa700 1 -- 172.21.15.123:0/1582849722 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fc01c018030 con 0x7fc020154950 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.693+0000 7fc0251ee700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc004066f30 0x7fc0040693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.694+0000 7fc0251ee700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc004066f30 0x7fc0040693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc010007bc0 tx=0x7fc010009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:20.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.698+0000 7fc0157fa700 1 -- 172.21.15.123:0/1582849722 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fc0200b0180 con 0x7fc020154950 2023-10-06T04:33:20.878 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.875+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc0200b0180 con 0x7fc004066f30 2023-10-06T04:33:20.880 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.878+0000 7fc0157fa700 1 -- 172.21.15.123:0/1582849722 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+72779 (secure 0 0 0) 0x7fc0200b0180 con 0x7fc004066f30 2023-10-06T04:33:20.881 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:20.882 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc004066f30 msgr2=0x7fc0040693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.883 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc004066f30 0x7fc0040693e0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fc010007bc0 tx=0x7fc010009040 comp rx=0 tx=0).stop 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc020154950 msgr2=0x7fc020168510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc020154950 0x7fc020168510 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7fc01c011b10 tx=0x7fc01c0091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 shutdown_connections 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fc004066f30 0x7fc0040693e0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fc020168cb0 0x7fc0200c5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fc00c0051c0 0x7fc02016aab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 --2- 172.21.15.123:0/1582849722 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fc020154950 0x7fc020168510 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 >> 172.21.15.123:0/1582849722 conn(0x7fc0200c9c90 msgr2=0x7fc020151350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 shutdown_connections 2023-10-06T04:33:20.884 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:20.880+0000 7fc027452700 1 -- 172.21.15.123:0/1582849722 wait complete. 2023-10-06T04:33:20.885 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:33:20.894 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":98,"stamp":"2023-10-06T04:33:18.910726+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2383720,"kb_used_data":5416,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591000,"statfs":{"total":773094113280,"available":770653184000,"internally_reserved":0,"allocated":5545984,"data_stored":3068256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-6291456,"num_objects":-2,"num_object_clones":0,"num_object_copies":-4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-2,"num_whiteouts":0,"num_read":-16,"num_read_kb":-9,"num_write":-8,"num_write_kb":-6144,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.466508"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:11.375350+0000","last_change":"2023-10-06T04:31:29.038862+0000","last_active":"2023-10-06T04:33:11.375350+0000","last_peered":"2023-10-06T04:33:11.375350+0000","last_clean":"2023-10-06T04:33:11.375350+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:33:11.375350+0000","last_undegraded":"2023-10-06T04:33:11.375350+0000","last_fullsized":"2023-10-06T04:33:11.375350+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:30:08.702228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:11.370792+0000","last_change":"2023-10-06T04:31:29.020237+0000","last_active":"2023-10-06T04:33:11.370792+0000","last_peered":"2023-10-06T04:33:11.370792+0000","last_clean":"2023-10-06T04:33:11.370792+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:33:11.370792+0000","last_undegraded":"2023-10-06T04:33:11.370792+0000","last_fullsized":"2023-10-06T04:33:11.370792+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:08:04.548004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":35,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:10.447047+0000","last_change":"2023-10-06T04:31:29.052574+0000","last_active":"2023-10-06T04:33:10.447047+0000","last_peered":"2023-10-06T04:33:10.447047+0000","last_clean":"2023-10-06T04:33:10.447047+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:33:10.447047+0000","last_undegraded":"2023-10-06T04:33:10.447047+0000","last_fullsized":"2023-10-06T04:33:10.447047+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":32,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:11.370932+0000","last_change":"2023-10-06T04:31:29.019927+0000","last_active":"2023-10-06T04:33:11.370932+0000","last_peered":"2023-10-06T04:33:11.370932+0000","last_clean":"2023-10-06T04:33:11.370932+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:33:11.370932+0000","last_undegraded":"2023-10-06T04:33:11.370932+0000","last_fullsized":"2023-10-06T04:33:11.370932+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T07:59:12.913215+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"15'117","reported_seq":186,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:10.453962+0000","last_change":"2023-10-06T04:31:23.082498+0000","last_active":"2023-10-06T04:33:10.453962+0000","last_peered":"2023-10-06T04:33:10.453962+0000","last_clean":"2023-10-06T04:33:10.453962+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:33:10.453962+0000","last_undegraded":"2023-10-06T04:33:10.453962+0000","last_fullsized":"2023-10-06T04:33:10.453962+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:21.875790+0000","last_clean_scrub_stamp":"2023-10-06T04:31:21.875790+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:55:56.478431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:10.453949+0000","last_change":"2023-10-06T04:31:29.020188+0000","last_active":"2023-10-06T04:33:10.453949+0000","last_peered":"2023-10-06T04:33:10.453949+0000","last_clean":"2023-10-06T04:33:10.453949+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:33:10.453949+0000","last_undegraded":"2023-10-06T04:33:10.453949+0000","last_fullsized":"2023-10-06T04:33:10.453949+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T08:51:49.130638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:10.455791+0000","last_change":"2023-10-06T04:31:29.038940+0000","last_active":"2023-10-06T04:33:10.455791+0000","last_peered":"2023-10-06T04:33:10.455791+0000","last_clean":"2023-10-06T04:33:10.455791+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:33:10.455791+0000","last_undegraded":"2023-10-06T04:33:10.455791+0000","last_fullsized":"2023-10-06T04:33:10.455791+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:46:47.264684+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":33,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:11.373015+0000","last_change":"2023-10-06T04:31:29.019092+0000","last_active":"2023-10-06T04:33:11.373015+0000","last_peered":"2023-10-06T04:33:11.373015+0000","last_clean":"2023-10-06T04:33:11.373015+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:33:11.373015+0000","last_undegraded":"2023-10-06T04:33:11.373015+0000","last_fullsized":"2023-10-06T04:33:11.373015+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:17:17.620965+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":34,"reported_epoch":45,"state":"active+clean","last_fresh":"2023-10-06T04:33:10.450555+0000","last_change":"2023-10-06T04:31:29.019039+0000","last_active":"2023-10-06T04:33:10.450555+0000","last_peered":"2023-10-06T04:33:10.450555+0000","last_clean":"2023-10-06T04:33:10.450555+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:33:10.450555+0000","last_undegraded":"2023-10-06T04:33:10.450555+0000","last_fullsized":"2023-10-06T04:33:10.450555+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:31:28.005221+0000","last_clean_scrub_stamp":"2023-10-06T04:31:28.005221+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:41:24.131534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1138688,"data_stored":826308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.23000000000000001,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":4.4790000000000001,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.73399999999999999,"15min":0.73399999999999999},"min":{"1min":0.28100000000000003,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":4.4269999999999996,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.47399999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.752,"5min":0.752,"15min":0.752},"last":0.628},{"interface":"front","average":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.59299999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.66500000000000004,"15min":0.66500000000000004},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":2.6429999999999998,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.57499999999999996}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.86799999999999999,"5min":0.86799999999999999,"15min":0.86799999999999999},"last":0.496},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.66800000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.34200000000000003,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.91700000000000004,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.68899999999999995}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.93500000000000005,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.70099999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.61799999999999999,"15min":0.61799999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.873,"5min":0.873,"15min":0.873},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.65500000000000003}]}]},{"osd":6,"up_from":13,"seq":55834574874,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.51600000000000001,"5min":0.51600000000000001,"15min":0.51600000000000001},"min":{"1min":0.246,"5min":0.246,"15min":0.246},"max":{"1min":0.85799999999999998,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.495,"5min":0.495,"15min":0.495},"min":{"1min":0.28999999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.55400000000000005}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.69199999999999995},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":3.7749999999999999,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.54000000000000004,"5min":0.54000000000000004,"15min":0.54000000000000004},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.80500000000000005,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.63600000000000001}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58699999999999997,"15min":0.58699999999999997},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.94599999999999995,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.90600000000000003,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.65600000000000003}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":3.6019999999999999,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.82399999999999995,"5min":0.82399999999999995,"15min":0.82399999999999995},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":5.5579999999999998,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.66600000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.626,"5min":0.626,"15min":0.626},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"min":{"1min":0.27100000000000002,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.70699999999999996},"last":0.60499999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.622},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"last":0.68500000000000005}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.56899999999999995,"15min":0.56899999999999995},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.74399999999999999,"5min":0.74399999999999999,"15min":0.74399999999999999},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":0.83499999999999996},"last":0.70499999999999996}]}]},{"osd":1,"up_from":13,"seq":55834574875,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"min":{"1min":0.224,"5min":0.224,"15min":0.224},"max":{"1min":4.1559999999999997,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.20999999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":3.9430000000000001,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.36299999999999999}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.23799999999999999,"5min":0.23799999999999999,"15min":0.23799999999999999},"max":{"1min":4.4000000000000004,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.39000000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.25800000000000001,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":4.2309999999999999,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.40200000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.60999999999999999,"15min":0.60999999999999999},"min":{"1min":0.30499999999999999,"5min":0.30499999999999999,"15min":0.30499999999999999},"max":{"1min":4.2560000000000002,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"min":{"1min":0.26900000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":3.8639999999999999,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.41499999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":4.0960000000000001,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.83699999999999997,"15min":0.83699999999999997},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":4.3040000000000003,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.73499999999999999}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":3.919,"5min":3.919,"15min":3.919},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":4.0229999999999997,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.72399999999999998}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.79400000000000004,"15min":0.79400000000000004},"min":{"1min":0.439,"5min":0.439,"15min":0.439},"max":{"1min":3.9649999999999999,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":4.0650000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.57399999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":4.2809999999999997,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":3.895,"5min":3.895,"15min":3.895},"last":0.60999999999999999}]}]},{"osd":0,"up_from":13,"seq":55834574875,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1138688,"data_stored":826308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.41199999999999998},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":1.633,"5min":1.633,"15min":1.633},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":1.494,"5min":1.494,"15min":1.494},"last":0.372}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.47099999999999997,"15min":0.47099999999999997},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":1.7150000000000001,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.30099999999999999,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":1.387,"5min":1.387,"15min":1.387},"last":0.41799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":1.345,"5min":1.345,"15min":1.345},"last":0.32600000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":1.738,"5min":1.738,"15min":1.738},"last":0.34999999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.62,"15min":0.62},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.413,"5min":1.413,"15min":1.413},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.64700000000000002,"15min":0.64700000000000002},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.3200000000000001,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.45900000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.69599999999999995,"15min":0.69599999999999995},"min":{"1min":0.41899999999999998,"5min":0.41899999999999998,"15min":0.41899999999999998},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.622,"5min":0.622,"15min":0.622},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.444,"5min":1.444,"15min":1.444},"last":0.58799999999999997}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.258,"5min":1.258,"15min":1.258},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.69799999999999995,"15min":0.69799999999999995},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":1.663,"5min":1.663,"15min":1.663},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.48599999999999999},"max":{"1min":1.468,"5min":1.468,"15min":1.468},"last":0.496},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.50900000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35099999999999998,"5min":0.35099999999999998,"15min":0.35099999999999998},"min":{"1min":0.20699999999999999,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.86199999999999999,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.33100000000000002},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"min":{"1min":0.219,"5min":0.219,"15min":0.219},"max":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.32100000000000001}]},{"osd":1,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83999999999999997,"5min":0.83999999999999997,"15min":0.83999999999999997},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":8.8170000000000002,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.307},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.84999999999999998,"15min":0.84999999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":8.3330000000000002,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.37}]},{"osd":3,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":8.7420000000000009,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.877,"5min":0.877,"15min":0.877},"min":{"1min":0.27500000000000002,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":8.3640000000000008,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.29299999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.58299999999999996,"15min":0.58299999999999996},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.88300000000000001,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":8.7870000000000008,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.53600000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.57299999999999995,"15min":0.57299999999999995},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.46300000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40500000000000003},"max":{"1min":8.8420000000000005,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.59999999999999998,"15min":0.59999999999999998},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"last":0.434}]},{"osd":7,"last update":"Fri Oct 6 04:32:22 2023","interfaces":[{"interface":"back","average":{"1min":0.58999999999999997,"5min":0.58999999999999997,"15min":0.58999999999999997},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"last":0.42699999999999999},{"interface":"front","average":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":8.8670000000000009,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.51300000000000001}]}]},{"osd":3,"up_from":13,"seq":55834574875,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.59899999999999998,"15min":0.59899999999999998},"min":{"1min":0.22800000000000001,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":4.5439999999999996,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.29999999999999999},{"interface":"front","average":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"min":{"1min":0.24399999999999999,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":1.1220000000000001,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.315}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"min":{"1min":0.22900000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.26800000000000002},{"interface":"front","average":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"min":{"1min":0.251,"5min":0.251,"15min":0.251},"max":{"1min":4.5869999999999997,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.34100000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.44,"15min":0.44},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":1.1459999999999999,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.35399999999999998},{"interface":"front","average":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"min":{"1min":0.28899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.33200000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61299999999999999,"15min":0.61299999999999999},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.1990000000000001,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.57199999999999995,"15min":0.57199999999999995},"min":{"1min":0.378,"5min":0.378,"15min":0.378},"max":{"1min":1.089,"5min":1.089,"15min":1.089},"last":0.504}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.57499999999999996,"15min":0.57499999999999996},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.03,"5min":1.03,"15min":1.03},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.81799999999999995,"15min":0.81799999999999995},"min":{"1min":0.42199999999999999,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":4.6219999999999999,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.44}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":4.6509999999999998,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65400000000000003,"15min":0.65400000000000003},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.1739999999999999,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.46300000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.54000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834574876,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":4.3780000000000001,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.625},{"interface":"front","average":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.42299999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.51100000000000001,"15min":0.51100000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.65900000000000003,"5min":0.65900000000000003,"15min":0.65900000000000003},"last":0.496},{"interface":"front","average":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.69099999999999995,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.55600000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.52200000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.79700000000000004,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58199999999999996,"15min":0.58199999999999996},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.47299999999999998},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.59799999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.78500000000000003,"15min":0.78500000000000003},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":4.452,"5min":4.452,"15min":4.452},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.65700000000000003,"15min":0.65700000000000003},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.40899999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.72299999999999998,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.53400000000000003},"min":{"1min":0.29299999999999998,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.52100000000000002}]},{"osd":6,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.70899999999999996,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.46200000000000002},{"interface":"front","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.35399999999999998,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.64900000000000002}]},{"osd":7,"last update":"Fri Oct 6 04:32:21 2023","interfaces":[{"interface":"back","average":{"1min":0.56699999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.75700000000000001,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.60799999999999998,"15min":0.60799999999999998},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.74299999999999999,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.57799999999999996}]}]},{"osd":5,"up_from":13,"seq":55834574875,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":1.107,"5min":1.107,"15min":1.107},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":12.130000000000001,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":1.2030000000000001,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.51800000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":1.2250000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.68400000000000005}]},{"osd":2,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.54500000000000004},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.236,"5min":1.236,"15min":1.236},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.58799999999999997,"15min":0.58799999999999997},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":1.298,"5min":1.298,"15min":1.298},"last":0.42099999999999999}]},{"osd":3,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":1.4159999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.64300000000000002},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":1.488,"5min":1.488,"15min":1.488},"last":0.59799999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.502,"15min":0.502},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.70399999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.60199999999999998,"15min":0.60199999999999998},"min":{"1min":0.38700000000000001,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.42899999999999999,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.626}]},{"osd":7,"last update":"Fri Oct 6 04:32:23 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.61699999999999999},"min":{"1min":0.45400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.88800000000000001,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.626,"5min":0.626,"15min":0.626},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.64200000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:33:20.900 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.900 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.900 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.900 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.901 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-10-06T04:31:21.875790+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=21, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.901 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.901 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.901 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.901 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2023-10-06T04:31:28.005221+0000 time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=31, tm_sec=28, tm_wday=4, tm_yday=279, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=10, tm_mday=6, tm_hour=4, tm_min=33, tm_sec=15, tm_wday=4, tm_yday=279, tm_isdst=0) 2023-10-06T04:33:20.902 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-10-06T04:33:33.635 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:33.639 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:40.903 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-10-06T04:33:41.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.037+0000 7fe233913700 1 Processor -- start 2023-10-06T04:33:41.038 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.037+0000 7fe233913700 1 -- start start 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe233913700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe22c0688c0 0x7fe22c0c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe233913700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c158f70 0x7fe22c1593f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe233913700 1 -- --> v1:172.21.15.177:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe22c069a30 con 0x7fe22c068c90 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe233913700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe22c06a890 con 0x7fe22c0688c0 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe233913700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c159930 con 0x7fe22c158f70 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe22c0688c0 0x7fe22c0c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7fe22c0688c0 0x7fe22c0c6dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48454/0 (socket says 172.21.15.123:48454) 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 -- 172.21.15.123:0/483340173 learned_addr learned my addr 172.21.15.123:0/483340173 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:41.041 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 -- 172.21.15.123:0/483340173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c158f70 msgr2=0x7fe22c1593f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe231eb0700 1 --2- 172.21.15.123:0/483340173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c158f70 0x7fe22c1593f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 --2- 172.21.15.123:0/483340173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c158f70 0x7fe22c1593f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 -- 172.21.15.123:0/483340173 >> v1:172.21.15.177:6789/0 conn(0x7fe22c068c90 legacy=0x7fe22c06e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 -- 172.21.15.123:0/483340173 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe22c069c80 con 0x7fe22c0688c0 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe2316af700 1 --2- 172.21.15.123:0/483340173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c0688c0 0x7fe22c0c6dd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe2280099b0 tx=0x7fe228009250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0944b120d4b3f3b server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.038+0000 7fe223fff700 1 -- 172.21.15.123:0/483340173 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800b820 con 0x7fe22c0688c0 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe223fff700 1 -- 172.21.15.123:0/483340173 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22800b980 con 0x7fe22c0688c0 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe223fff700 1 -- 172.21.15.123:0/483340173 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22800bd20 con 0x7fe22c0688c0 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 -- 172.21.15.123:0/483340173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c0688c0 msgr2=0x7fe22c0c6dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.042 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 --2- 172.21.15.123:0/483340173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c0688c0 0x7fe22c0c6dd0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7fe2280099b0 tx=0x7fe228009250 comp rx=0 tx=0).stop 2023-10-06T04:33:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 -- 172.21.15.123:0/483340173 shutdown_connections 2023-10-06T04:33:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 --2- 172.21.15.123:0/483340173 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c158f70 0x7fe22c1593f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 --2- 172.21.15.123:0/483340173 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c0688c0 0x7fe22c0c6dd0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 -- 172.21.15.123:0/483340173 >> 172.21.15.123:0/483340173 conn(0x7fe22c0c9c90 msgr2=0x7fe22c06da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 -- 172.21.15.123:0/483340173 shutdown_connections 2023-10-06T04:33:41.043 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.039+0000 7fe233913700 1 -- 172.21.15.123:0/483340173 wait complete. 2023-10-06T04:33:41.045 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.040+0000 7fe233913700 1 Processor -- start 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.040+0000 7fe233913700 1 -- start start 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe233913700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c0688c0 0x7fe22c0ae5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe233913700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c068c90 0x7fe22c1569b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe233913700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c158f70 0x7fe22c15b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe233913700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c15b160 con 0x7fe22c158f70 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe233913700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c15acd0 con 0x7fe22c068c90 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe233913700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe22c0aeb10 con 0x7fe22c0688c0 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe231eb0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c158f70 0x7fe22c15b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe231eb0700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c158f70 0x7fe22c15b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48462/0 (socket says 172.21.15.123:48462) 2023-10-06T04:33:41.046 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe231eb0700 1 -- 172.21.15.123:0/3469981760 learned_addr learned my addr 172.21.15.123:0/3469981760 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe2316af700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c0688c0 0x7fe22c0ae5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.041+0000 7fe230eae700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c068c90 0x7fe22c1569b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe231eb0700 1 -- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c0688c0 msgr2=0x7fe22c0ae5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe231eb0700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c0688c0 0x7fe22c0ae5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe231eb0700 1 -- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c068c90 msgr2=0x7fe22c1569b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe231eb0700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c068c90 0x7fe22c1569b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe231eb0700 1 -- 172.21.15.123:0/3469981760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2280094f0 con 0x7fe22c158f70 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe231eb0700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c158f70 0x7fe22c15b6d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe22400cdb0 tx=0x7fe2240077d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe2217fa700 1 -- 172.21.15.123:0/3469981760 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe22400a9d0 con 0x7fe22c158f70 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe2217fa700 1 -- 172.21.15.123:0/3469981760 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe22400ab30 con 0x7fe22c158f70 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe22c156ef0 con 0x7fe22c158f70 2023-10-06T04:33:41.047 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe2217fa700 1 -- 172.21.15.123:0/3469981760 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe224017430 con 0x7fe22c158f70 2023-10-06T04:33:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.042+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe22c15a3a0 con 0x7fe22c158f70 2023-10-06T04:33:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.043+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe22c0c49c0 con 0x7fe22c158f70 2023-10-06T04:33:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.043+0000 7fe2217fa700 1 -- 172.21.15.123:0/3469981760 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7fe224003020 con 0x7fe22c158f70 2023-10-06T04:33:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.044+0000 7fe2217fa700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe210066f30 0x7fe2100693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.044+0000 7fe2217fa700 1 -- 172.21.15.123:0/3469981760 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7fe224013070 con 0x7fe22c158f70 2023-10-06T04:33:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.044+0000 7fe2316af700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe210066f30 0x7fe2100693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.048 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.044+0000 7fe2316af700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe210066f30 0x7fe2100693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe2280099b0 tx=0x7fe228016000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.049 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.048+0000 7fe2217fa700 1 -- 172.21.15.123:0/3469981760 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7fe224051ac0 con 0x7fe22c158f70 2023-10-06T04:33:41.228 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.226+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 --> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe22c15ae40 con 0x7fe210066f30 2023-10-06T04:33:41.230 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.229+0000 7fe2217fa700 1 -- 172.21.15.123:0/3469981760 <== mgr.4104 v2:172.21.15.123:6832/119418 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+73409 (secure 0 0 0) 0x7fe22c15ae40 con 0x7fe210066f30 2023-10-06T04:33:41.231 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:41.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe210066f30 msgr2=0x7fe2100693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe210066f30 0x7fe2100693e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fe2280099b0 tx=0x7fe228016000 comp rx=0 tx=0).stop 2023-10-06T04:33:41.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c158f70 msgr2=0x7fe22c15b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c158f70 0x7fe22c15b6d0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fe22400cdb0 tx=0x7fe2240077d0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 shutdown_connections 2023-10-06T04:33:41.233 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7fe210066f30 0x7fe2100693e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7fe22c158f70 0x7fe22c15b6d0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7fe22c0688c0 0x7fe22c0ae5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.231+0000 7fe233913700 1 --2- 172.21.15.123:0/3469981760 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7fe22c068c90 0x7fe22c1569b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.232+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 >> 172.21.15.123:0/3469981760 conn(0x7fe22c0c9c90 msgr2=0x7fe22c157780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.232+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 shutdown_connections 2023-10-06T04:33:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.232+0000 7fe233913700 1 -- 172.21.15.123:0/3469981760 wait complete. 2023-10-06T04:33:41.234 INFO:teuthology.orchestra.run.smithi123.stderr:dumped all 2023-10-06T04:33:41.244 INFO:teuthology.orchestra.run.smithi123.stdout:{"pg_ready":true,"pg_map":{"version":109,"stamp":"2023-10-06T04:33:40.915378+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":18,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":2383720,"kb_used_data":5416,"kb_used_omap":0,"kb_used_meta":2378240,"kb_avail":752591000,"statfs":{"total":773094113280,"available":770653184000,"internally_reserved":0,"allocated":5545984,"data_stored":3068256,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":2435317760},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002542"},"pg_stats":[{"pgid":"2.4","version":"0'0","reported_seq":44,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:21.172744+0000","last_change":"2023-10-06T04:33:21.172721+0000","last_active":"2023-10-06T04:33:21.172744+0000","last_peered":"2023-10-06T04:33:21.172744+0000","last_clean":"2023-10-06T04:33:21.172744+0000","last_became_active":"2023-10-06T04:31:29.038530+0000","last_became_peered":"2023-10-06T04:31:29.038530+0000","last_unstale":"2023-10-06T04:33:21.172744+0000","last_undegraded":"2023-10-06T04:33:21.172744+0000","last_fullsized":"2023-10-06T04:33:21.172744+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:21.172630+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:21.172630+0000","last_clean_scrub_stamp":"2023-10-06T04:33:21.172630+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:28.830296+0000","scrub_duration":0.001175082,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":43,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:25.137769+0000","last_change":"2023-10-06T04:33:25.137714+0000","last_active":"2023-10-06T04:33:25.137769+0000","last_peered":"2023-10-06T04:33:25.137769+0000","last_clean":"2023-10-06T04:33:25.137769+0000","last_became_active":"2023-10-06T04:31:29.019891+0000","last_became_peered":"2023-10-06T04:31:29.019891+0000","last_unstale":"2023-10-06T04:33:25.137769+0000","last_undegraded":"2023-10-06T04:33:25.137769+0000","last_fullsized":"2023-10-06T04:33:25.137769+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:25.137579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:25.137579+0000","last_clean_scrub_stamp":"2023-10-06T04:33:25.137579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:20.396562+0000","scrub_duration":0.0021208590000000001,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":46,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:18.220834+0000","last_change":"2023-10-06T04:33:18.220813+0000","last_active":"2023-10-06T04:33:18.220834+0000","last_peered":"2023-10-06T04:33:18.220834+0000","last_clean":"2023-10-06T04:33:18.220834+0000","last_became_active":"2023-10-06T04:31:29.052170+0000","last_became_peered":"2023-10-06T04:31:29.052170+0000","last_unstale":"2023-10-06T04:33:18.220834+0000","last_undegraded":"2023-10-06T04:33:18.220834+0000","last_fullsized":"2023-10-06T04:33:18.220834+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:18.220730+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:18.220730+0000","last_clean_scrub_stamp":"2023-10-06T04:33:18.220730+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:16:53.211934+0000","scrub_duration":0.001278724,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":43,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:22.090208+0000","last_change":"2023-10-06T04:33:22.090158+0000","last_active":"2023-10-06T04:33:22.090208+0000","last_peered":"2023-10-06T04:33:22.090208+0000","last_clean":"2023-10-06T04:33:22.090208+0000","last_became_active":"2023-10-06T04:31:29.019558+0000","last_became_peered":"2023-10-06T04:31:29.019558+0000","last_unstale":"2023-10-06T04:33:22.090208+0000","last_undegraded":"2023-10-06T04:33:22.090208+0000","last_fullsized":"2023-10-06T04:33:22.090208+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:22.090048+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:22.090048+0000","last_clean_scrub_stamp":"2023-10-06T04:33:22.090048+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:08.408445+0000","scrub_duration":0.0021293779999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"46'119","reported_seq":199,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:23.224991+0000","last_change":"2023-10-06T04:33:23.224964+0000","last_active":"2023-10-06T04:33:23.224991+0000","last_peered":"2023-10-06T04:33:23.224991+0000","last_clean":"2023-10-06T04:33:23.224991+0000","last_became_active":"2023-10-06T04:31:23.081615+0000","last_became_peered":"2023-10-06T04:31:23.081615+0000","last_unstale":"2023-10-06T04:33:23.224991+0000","last_undegraded":"2023-10-06T04:33:23.224991+0000","last_fullsized":"2023-10-06T04:33:23.224991+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"46'119","last_scrub_stamp":"2023-10-06T04:33:23.224860+0000","last_deep_scrub":"46'119","last_deep_scrub_stamp":"2023-10-06T04:33:23.224860+0000","last_clean_scrub_stamp":"2023-10-06T04:33:23.224860+0000","objects_scrubbed":2,"log_size":119,"log_dups_size":0,"ondisk_log_size":119,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T09:24:28.840212+0000","scrub_duration":0.0055203479999999996,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":44,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:25.273589+0000","last_change":"2023-10-06T04:33:25.273554+0000","last_active":"2023-10-06T04:33:25.273589+0000","last_peered":"2023-10-06T04:33:25.273589+0000","last_clean":"2023-10-06T04:33:25.273589+0000","last_became_active":"2023-10-06T04:31:29.019858+0000","last_became_peered":"2023-10-06T04:31:29.019858+0000","last_unstale":"2023-10-06T04:33:25.273589+0000","last_undegraded":"2023-10-06T04:33:25.273589+0000","last_fullsized":"2023-10-06T04:33:25.273589+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:25.273376+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:25.273376+0000","last_clean_scrub_stamp":"2023-10-06T04:33:25.273376+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T06:50:11.591773+0000","scrub_duration":0.0018030679999999999,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":44,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:28.286107+0000","last_change":"2023-10-06T04:33:28.286077+0000","last_active":"2023-10-06T04:33:28.286107+0000","last_peered":"2023-10-06T04:33:28.286107+0000","last_clean":"2023-10-06T04:33:28.286107+0000","last_became_active":"2023-10-06T04:31:29.038635+0000","last_became_peered":"2023-10-06T04:31:29.038635+0000","last_unstale":"2023-10-06T04:33:28.286107+0000","last_undegraded":"2023-10-06T04:33:28.286107+0000","last_fullsized":"2023-10-06T04:33:28.286107+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:28.285970+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:28.285970+0000","last_clean_scrub_stamp":"2023-10-06T04:33:28.285970+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T14:58:23.544953+0000","scrub_duration":0.001872571,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":44,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:23.155607+0000","last_change":"2023-10-06T04:33:23.155567+0000","last_active":"2023-10-06T04:33:23.155607+0000","last_peered":"2023-10-06T04:33:23.155607+0000","last_clean":"2023-10-06T04:33:23.155607+0000","last_became_active":"2023-10-06T04:31:29.018832+0000","last_became_peered":"2023-10-06T04:31:29.018832+0000","last_unstale":"2023-10-06T04:33:23.155607+0000","last_undegraded":"2023-10-06T04:33:23.155607+0000","last_fullsized":"2023-10-06T04:33:23.155607+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:23.155452+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:23.155452+0000","last_clean_scrub_stamp":"2023-10-06T04:33:23.155452+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T13:12:36.105824+0000","scrub_duration":0.0021715990000000002,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":45,"reported_epoch":46,"state":"active+clean","last_fresh":"2023-10-06T04:33:24.223367+0000","last_change":"2023-10-06T04:33:24.223333+0000","last_active":"2023-10-06T04:33:24.223367+0000","last_peered":"2023-10-06T04:33:24.223367+0000","last_clean":"2023-10-06T04:33:24.223367+0000","last_became_active":"2023-10-06T04:31:29.018774+0000","last_became_peered":"2023-10-06T04:31:29.018774+0000","last_unstale":"2023-10-06T04:33:24.223367+0000","last_undegraded":"2023-10-06T04:33:24.223367+0000","last_fullsized":"2023-10-06T04:33:24.223367+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-10-06T04:33:24.223234+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-10-06T04:33:24.223234+0000","last_clean_scrub_stamp":"2023-10-06T04:33:24.223234+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-10-07T12:14:31.880901+0000","scrub_duration":0.001562187,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":119,"ondisk_log_size":119,"up":2,"acting":2,"num_store_stats":2},{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":7}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574879,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1138688,"data_stored":826308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.48999999999999999,"5min":0.65900000000000003,"15min":0.68700000000000006},"min":{"1min":0.24099999999999999,"5min":0.23000000000000001,"15min":0.23000000000000001},"max":{"1min":0.65100000000000002,"5min":4.4790000000000001,"15min":4.4790000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.46600000000000003,"5min":0.68000000000000005,"15min":0.71599999999999997},"min":{"1min":0.311,"5min":0.28100000000000003,"15min":0.28100000000000003},"max":{"1min":0.67100000000000004,"5min":4.4269999999999996,"15min":4.4269999999999996},"last":0.68700000000000006}]},{"osd":1,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"min":{"1min":0.39200000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.68600000000000005,"5min":0.752,"15min":0.752},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.57599999999999996,"15min":0.57199999999999995},"min":{"1min":0.41999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.90500000000000003,"5min":0.90500000000000003,"15min":0.90500000000000003},"last":0.58299999999999996}]},{"osd":2,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.63800000000000001,"15min":0.65600000000000003},"min":{"1min":0.32800000000000001,"5min":0.30099999999999999,"15min":0.30099999999999999},"max":{"1min":0.80400000000000005,"5min":2.6429999999999998,"15min":2.6429999999999998},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55300000000000005,"15min":0.55500000000000005},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.90900000000000003,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.57199999999999995}]},{"osd":3,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.84099999999999997,"5min":0.629,"15min":0.59299999999999997},"min":{"1min":0.38300000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":6.2560000000000002,"5min":6.2560000000000002,"15min":6.2560000000000002},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.872,"5min":0.65700000000000003,"15min":0.621},"min":{"1min":0.44900000000000001,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":6.2329999999999997,"5min":6.2329999999999997,"15min":6.2329999999999997},"last":0.64900000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.55900000000000005,"15min":0.56399999999999995},"min":{"1min":0.36899999999999999,"5min":0.34200000000000003,"15min":0.34200000000000003},"max":{"1min":0.85599999999999998,"5min":0.85599999999999998,"15min":0.85599999999999998},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.55600000000000005,"15min":0.55400000000000005},"min":{"1min":0.41399999999999998,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.72499999999999998,"5min":0.91700000000000004,"15min":0.91700000000000004},"last":0.59499999999999997}]},{"osd":5,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.54500000000000004,"15min":0.54300000000000004},"min":{"1min":0.40500000000000003,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.71099999999999997,"5min":0.93500000000000005,"15min":0.93500000000000005},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.622,"5min":0.58999999999999997,"15min":0.58399999999999996},"min":{"1min":0.438,"5min":0.36799999999999999,"15min":0.36799999999999999},"max":{"1min":0.95899999999999996,"5min":0.95899999999999996,"15min":0.95899999999999996},"last":0.62}]},{"osd":6,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.621,"15min":0.61899999999999999},"min":{"1min":0.45300000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.625,"15min":0.627},"min":{"1min":0.45900000000000002,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.76400000000000001,"5min":0.95599999999999996,"15min":0.95599999999999996},"last":0.69899999999999995}]}]},{"osd":6,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53900000000000003,"5min":0.52000000000000002,"15min":0.51700000000000002},"min":{"1min":0.38200000000000001,"5min":0.246,"15min":0.246},"max":{"1min":0.78200000000000003,"5min":0.85799999999999998,"15min":0.85799999999999998},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.5,"15min":0.496},"min":{"1min":0.35299999999999998,"5min":0.28999999999999998,"15min":0.28999999999999998},"max":{"1min":0.79100000000000004,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.66600000000000004,"15min":0.68300000000000005},"min":{"1min":0.36799999999999999,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.71699999999999997,"5min":3.7749999999999999,"15min":3.7749999999999999},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.54100000000000004,"15min":0.54000000000000004},"min":{"1min":0.34899999999999998,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.73199999999999998,"5min":0.80500000000000005,"15min":0.80500000000000005},"last":0.79700000000000004}]},{"osd":2,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.36399999999999999,"5min":0.309,"15min":0.309},"max":{"1min":0.90400000000000003,"5min":0.94599999999999995,"15min":0.94599999999999995},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.40600000000000003,"5min":0.38,"15min":0.38},"max":{"1min":0.86399999999999999,"5min":0.90600000000000003,"15min":0.90600000000000003},"last":0.73099999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.68600000000000005,"15min":0.70699999999999996},"min":{"1min":0.41299999999999998,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.82099999999999995,"5min":3.6019999999999999,"15min":3.6019999999999999},"last":0.77200000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.77500000000000002,"15min":0.80700000000000005},"min":{"1min":0.45700000000000002,"5min":0.434,"15min":0.434},"max":{"1min":0.78400000000000003,"5min":5.5579999999999998,"15min":5.5579999999999998},"last":0.66900000000000004}]},{"osd":4,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.47499999999999998,"15min":0.47299999999999998},"min":{"1min":0.29599999999999999,"5min":0.25,"15min":0.25},"max":{"1min":0.79300000000000004,"5min":0.79300000000000004,"15min":0.79300000000000004},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.52700000000000002,"15min":0.52500000000000002},"min":{"1min":0.38400000000000001,"5min":0.27100000000000002,"15min":0.27100000000000002},"max":{"1min":0.88500000000000001,"5min":0.88500000000000001,"15min":0.88500000000000001},"last":0.47299999999999998}]},{"osd":5,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53100000000000003,"5min":0.53100000000000003,"15min":0.53100000000000003},"min":{"1min":0.378,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82999999999999996,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.56100000000000005,"15min":0.55700000000000005},"min":{"1min":0.40500000000000003,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.85999999999999999,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.502}]},{"osd":7,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57999999999999996,"15min":0.57199999999999995},"min":{"1min":0.44,"5min":0.315,"15min":0.315},"max":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"last":0.51200000000000001},{"interface":"front","average":{"1min":0.64900000000000002,"5min":0.61599999999999999,"15min":0.60999999999999999},"min":{"1min":0.46700000000000003,"5min":0.377,"15min":0.377},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":0.93300000000000005},"last":0.61199999999999999}]}]},{"osd":1,"up_from":13,"seq":55834574880,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.34100000000000003,"5min":0.50900000000000001,"15min":0.53700000000000003},"min":{"1min":0.255,"5min":0.224,"15min":0.224},"max":{"1min":0.53400000000000003,"5min":4.1559999999999997,"15min":4.1559999999999997},"last":0.42899999999999999},{"interface":"front","average":{"1min":0.33800000000000002,"5min":0.48699999999999999,"15min":0.51200000000000001},"min":{"1min":0.23599999999999999,"5min":0.20999999999999999,"15min":0.20999999999999999},"max":{"1min":0.59399999999999997,"5min":3.9430000000000001,"15min":3.9430000000000001},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.32300000000000001,"5min":0.50700000000000001,"15min":0.53800000000000003},"min":{"1min":0.23200000000000001,"5min":0.23200000000000001,"15min":0.23200000000000001},"max":{"1min":0.51000000000000001,"5min":4.4000000000000004,"15min":4.4000000000000004},"last":0.36199999999999999},{"interface":"front","average":{"1min":0.36399999999999999,"5min":0.54500000000000004,"15min":0.57499999999999996},"min":{"1min":0.30399999999999999,"5min":0.25800000000000001,"15min":0.25800000000000001},"max":{"1min":0.57299999999999995,"5min":4.2309999999999999,"15min":4.2309999999999999},"last":0.499}]},{"osd":3,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.38,"5min":0.56399999999999995,"15min":0.59399999999999997},"min":{"1min":0.29599999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.61299999999999999,"5min":4.2560000000000002,"15min":4.2560000000000002},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.37,"5min":0.52200000000000002,"15min":0.54700000000000004},"min":{"1min":0.27500000000000002,"5min":0.26900000000000002,"15min":0.26900000000000002},"max":{"1min":0.52400000000000002,"5min":3.8639999999999999,"15min":3.8639999999999999},"last":0.47599999999999998}]},{"osd":4,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.79200000000000004,"15min":0.82199999999999995},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83099999999999996,"5min":4.0960000000000001,"15min":4.0960000000000001},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.64700000000000002,"5min":0.79900000000000004,"15min":0.82399999999999995},"min":{"1min":0.42399999999999999,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.86199999999999999,"5min":4.3040000000000003,"15min":4.3040000000000003},"last":0.65200000000000002}]},{"osd":5,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.72799999999999998,"15min":0.749},"min":{"1min":0.441,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.93600000000000005,"5min":3.919,"15min":3.919},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.60299999999999998,"5min":0.755,"15min":0.78000000000000003},"min":{"1min":0.42599999999999999,"5min":0.42599999999999999,"15min":0.42599999999999999},"max":{"1min":0.755,"5min":4.0229999999999997,"15min":4.0229999999999997},"last":0.623}]},{"osd":6,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.60399999999999998,"5min":0.75600000000000001,"15min":0.78100000000000003},"min":{"1min":0.44700000000000001,"5min":0.439,"15min":0.439},"max":{"1min":0.81299999999999994,"5min":3.9649999999999999,"15min":3.9649999999999999},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.76800000000000002,"15min":0.79400000000000004},"min":{"1min":0.40899999999999997,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":0.79500000000000004,"5min":4.0650000000000004,"15min":4.0650000000000004},"last":0.60199999999999998}]},{"osd":7,"last update":"Fri Oct 6 04:33:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.80600000000000005,"15min":0.83199999999999996},"min":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.82599999999999996,"5min":4.2809999999999997,"15min":4.2809999999999997},"last":0.67900000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.75800000000000001,"15min":0.77800000000000002},"min":{"1min":0.47099999999999997,"5min":0.441,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":3.895,"15min":3.895},"last":0.63800000000000001}]}]},{"osd":0,"up_from":13,"seq":55834574879,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":298400,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94073440,"statfs":{"total":96636764160,"available":96331202560,"internally_reserved":0,"allocated":1138688,"data_stored":826308,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.33300000000000002,"5min":0.39600000000000002,"15min":0.40600000000000003},"min":{"1min":0.21099999999999999,"5min":0.21099999999999999,"15min":0.21099999999999999},"max":{"1min":0.5,"5min":1.633,"15min":1.633},"last":0.314},{"interface":"front","average":{"1min":0.33700000000000002,"5min":0.39300000000000002,"15min":0.40200000000000002},"min":{"1min":0.24099999999999999,"5min":0.24099999999999999,"15min":0.24099999999999999},"max":{"1min":0.50800000000000001,"5min":1.494,"15min":1.494},"last":0.46999999999999997}]},{"osd":2,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.38800000000000001,"5min":0.45400000000000001,"15min":0.46500000000000002},"min":{"1min":0.26300000000000001,"5min":0.26300000000000001,"15min":0.26300000000000001},"max":{"1min":0.55300000000000005,"5min":1.7150000000000001,"15min":1.7150000000000001},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.38400000000000001,"5min":0.434,"15min":0.442},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.28499999999999998},"max":{"1min":0.54400000000000004,"5min":1.387,"15min":1.387},"last":0.40300000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.35699999999999998,"5min":0.41299999999999998,"15min":0.42199999999999999},"min":{"1min":0.22500000000000001,"5min":0.22500000000000001,"15min":0.22500000000000001},"max":{"1min":0.56499999999999995,"5min":1.345,"15min":1.345},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.38500000000000001,"5min":0.47099999999999997,"15min":0.48499999999999999},"min":{"1min":0.25700000000000001,"5min":0.25700000000000001,"15min":0.25700000000000001},"max":{"1min":0.67500000000000004,"5min":1.738,"15min":1.738},"last":0.434}]},{"osd":4,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.60799999999999998,"15min":0.61599999999999999},"min":{"1min":0.38100000000000001,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.78200000000000003,"5min":1.413,"15min":1.413},"last":0.64600000000000002},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.628,"15min":0.64000000000000001},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.74299999999999999,"5min":1.3200000000000001,"15min":1.3200000000000001},"last":0.63400000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.67500000000000004,"15min":0.68899999999999995},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.82199999999999995,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.57899999999999996,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.86799999999999999,"5min":1.444,"15min":1.444},"last":0.58299999999999996}]},{"osd":6,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57799999999999996,"5min":0.64800000000000002,"15min":0.66000000000000003},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.88700000000000001,"5min":1.258,"15min":1.258},"last":0.57099999999999995},{"interface":"front","average":{"1min":0.59799999999999998,"5min":0.67800000000000005,"15min":0.69099999999999995},"min":{"1min":0.433,"5min":0.433,"15min":0.433},"max":{"1min":0.77000000000000002,"5min":1.663,"15min":1.663},"last":0.65800000000000003}]},{"osd":7,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.64600000000000002,"15min":0.65700000000000003},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.81200000000000006,"5min":1.468,"15min":1.468},"last":0.625},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.64700000000000002,"15min":0.65300000000000002},"min":{"1min":0.45200000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.78100000000000003,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.67200000000000004}]}]},{"osd":2,"up_from":13,"seq":55834574880,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:33:23 2023","interfaces":[{"interface":"back","average":{"1min":0.313,"5min":0.34300000000000003,"15min":0.34799999999999998},"min":{"1min":0.22,"5min":0.20699999999999999,"15min":0.20699999999999999},"max":{"1min":0.46400000000000002,"5min":0.86199999999999999,"15min":0.86199999999999999},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.315,"5min":0.35399999999999998,"15min":0.35999999999999999},"min":{"1min":0.23100000000000001,"5min":0.219,"15min":0.219},"max":{"1min":0.38200000000000001,"5min":0.54100000000000004,"15min":0.54100000000000004},"last":0.35999999999999999}]},{"osd":1,"last update":"Fri Oct 6 04:33:23 2023","interfaces":[{"interface":"back","average":{"1min":0.318,"5min":0.73499999999999999,"15min":0.80500000000000005},"min":{"1min":0.23599999999999999,"5min":0.23599999999999999,"15min":0.23599999999999999},"max":{"1min":0.44700000000000001,"5min":8.8170000000000002,"15min":8.8170000000000002},"last":0.379},{"interface":"front","average":{"1min":0.34699999999999998,"5min":0.749,"15min":0.81599999999999995},"min":{"1min":0.27000000000000002,"5min":0.251,"15min":0.251},"max":{"1min":0.501,"5min":8.3330000000000002,"15min":8.3330000000000002},"last":0.39300000000000002}]},{"osd":3,"last update":"Fri Oct 6 04:33:23 2023","interfaces":[{"interface":"back","average":{"1min":0.36299999999999999,"5min":0.78700000000000003,"15min":0.85799999999999998},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.48199999999999998,"5min":8.7420000000000009,"15min":8.7420000000000009},"last":0.41399999999999998},{"interface":"front","average":{"1min":0.38200000000000001,"5min":0.77800000000000002,"15min":0.84399999999999997},"min":{"1min":0.29299999999999998,"5min":0.27500000000000002,"15min":0.27500000000000002},"max":{"1min":0.46700000000000003,"5min":8.3640000000000008,"15min":8.3640000000000008},"last":0.435}]},{"osd":4,"last update":"Fri Oct 6 04:33:23 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.57699999999999996,"15min":0.58099999999999996},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.72299999999999998,"5min":0.88300000000000001,"15min":0.88300000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.97399999999999998,"15min":1.046},"min":{"1min":0.374,"5min":0.374,"15min":0.374},"max":{"1min":0.77900000000000003,"5min":8.7870000000000008,"15min":8.7870000000000008},"last":0.65600000000000003}]},{"osd":5,"last update":"Fri Oct 6 04:33:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.59499999999999997,"15min":0.60099999999999998},"min":{"1min":0.40000000000000002,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.79800000000000004,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.56699999999999995,"15min":0.57099999999999995},"min":{"1min":0.34300000000000003,"5min":0.34300000000000003,"15min":0.34300000000000003},"max":{"1min":0.78300000000000003,"5min":0.78300000000000003,"15min":0.78300000000000003},"last":0.61699999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:33:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.97799999999999998,"15min":1.0469999999999999},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.72699999999999998,"5min":8.8420000000000005,"15min":8.8420000000000005},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.59299999999999997,"15min":0.59699999999999998},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.84699999999999998,"5min":0.84699999999999998,"15min":0.84699999999999998},"last":0.57199999999999995}]},{"osd":7,"last update":"Fri Oct 6 04:33:23 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.58499999999999996,"15min":0.58799999999999997},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.82299999999999995},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.60099999999999998,"5min":1.022,"15min":1.0920000000000001},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.80200000000000005,"5min":8.8670000000000009,"15min":8.8670000000000009},"last":0.624}]}]},{"osd":3,"up_from":13,"seq":55834574879,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.30399999999999999,"5min":0.54000000000000004,"15min":0.57899999999999996},"min":{"1min":0.23999999999999999,"5min":0.22800000000000001,"15min":0.22800000000000001},"max":{"1min":0.438,"5min":4.5439999999999996,"15min":4.5439999999999996},"last":0.433},{"interface":"front","average":{"1min":0.35099999999999998,"5min":0.38300000000000001,"15min":0.38900000000000001},"min":{"1min":0.28599999999999998,"5min":0.24399999999999999,"15min":0.24399999999999999},"max":{"1min":0.46700000000000003,"5min":1.1220000000000001,"15min":1.1220000000000001},"last":0.46500000000000002}]},{"osd":1,"last update":"Fri Oct 6 04:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.32100000000000001,"5min":0.39400000000000002,"15min":0.40600000000000003},"min":{"1min":0.26300000000000001,"5min":0.22900000000000001,"15min":0.22900000000000001},"max":{"1min":0.48199999999999998,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.35299999999999998,"5min":0.55300000000000005,"15min":0.58699999999999997},"min":{"1min":0.28999999999999998,"5min":0.251,"15min":0.251},"max":{"1min":0.57599999999999996,"5min":4.5869999999999997,"15min":4.5869999999999997},"last":0.53600000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.377,"5min":0.42699999999999999,"15min":0.435},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.52400000000000002,"5min":1.1459999999999999,"15min":1.1459999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.39700000000000002,"5min":0.42299999999999999,"15min":0.42699999999999999},"min":{"1min":0.33200000000000002,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.51500000000000001,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.50700000000000001}]},{"osd":4,"last update":"Fri Oct 6 04:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.60099999999999998,"15min":0.60899999999999999},"min":{"1min":0.36399999999999999,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.73999999999999999,"5min":1.1990000000000001,"15min":1.1990000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.55500000000000005,"5min":0.56799999999999995,"15min":0.56999999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.64400000000000002,"5min":1.089,"15min":1.089},"last":0.63600000000000001}]},{"osd":5,"last update":"Fri Oct 6 04:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"min":{"1min":0.42899999999999999,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":0.71899999999999997,"5min":1.03,"15min":1.03},"last":0.57599999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.76700000000000002,"15min":0.80100000000000005},"min":{"1min":0.44,"5min":0.42199999999999999,"15min":0.42199999999999999},"max":{"1min":0.75900000000000001,"5min":4.6219999999999999,"15min":4.6219999999999999},"last":0.65800000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.76400000000000001,"15min":0.79400000000000004},"min":{"1min":0.44600000000000001,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.73499999999999999,"5min":4.6509999999999998,"15min":4.6509999999999998},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.63800000000000001,"15min":0.64800000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.79400000000000004,"5min":1.1739999999999999,"15min":1.1739999999999999},"last":0.624}]},{"osd":7,"last update":"Fri Oct 6 04:33:26 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.622,"15min":0.63400000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.73799999999999999,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.59199999999999997,"5min":0.627,"15min":0.63300000000000001},"min":{"1min":0.45500000000000002,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.69999999999999996,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.68400000000000005}]}]},{"osd":4,"up_from":13,"seq":55834574880,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.504,"5min":0.69799999999999995,"15min":0.72999999999999998},"min":{"1min":0.39300000000000002,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.68100000000000005,"5min":4.3780000000000001,"15min":4.3780000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.47099999999999997,"5min":0.45700000000000002,"15min":0.45500000000000002},"min":{"1min":0.34599999999999997,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.59099999999999997,"5min":0.60899999999999999,"15min":0.60899999999999999},"last":0.53400000000000003}]},{"osd":1,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"min":{"1min":0.39000000000000001,"5min":0.371,"15min":0.371},"max":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"last":0.626},{"interface":"front","average":{"1min":0.498,"5min":0.49299999999999999,"15min":0.49199999999999999},"min":{"1min":0.36699999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.59399999999999997,"5min":0.69099999999999995,"15min":0.69099999999999995},"last":0.65100000000000002}]},{"osd":2,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.52300000000000002},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.73199999999999998,"5min":0.79700000000000004,"15min":0.79700000000000004},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.56399999999999995,"5min":0.57799999999999996,"15min":0.57999999999999996},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.71299999999999997,"5min":0.85999999999999999,"15min":0.85999999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.72999999999999998,"15min":0.76600000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.64100000000000001,"5min":4.452,"15min":4.452},"last":0.61599999999999999},{"interface":"front","average":{"1min":0.505,"5min":0.626,"15min":0.64600000000000002},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.66900000000000004,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.66800000000000004}]},{"osd":5,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.502,"15min":0.50600000000000001},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.63700000000000001,"5min":0.72299999999999998,"15min":0.72299999999999998},"last":0.377},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.52000000000000002,"15min":0.52900000000000003},"min":{"1min":0.33000000000000002,"5min":0.29299999999999998,"15min":0.29299999999999998},"max":{"1min":0.61299999999999999,"5min":0.71799999999999997,"15min":0.71799999999999997},"last":0.43099999999999999}]},{"osd":6,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.52500000000000002,"15min":0.53300000000000003},"min":{"1min":0.34799999999999998,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.629,"5min":0.70899999999999996,"15min":0.70899999999999996},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.57499999999999996,"5min":0.59599999999999997,"15min":0.59999999999999998},"min":{"1min":0.36599999999999999,"5min":0.35399999999999998,"15min":0.35399999999999998},"max":{"1min":0.66500000000000004,"5min":0.77200000000000002,"15min":0.77200000000000002},"last":0.58599999999999997}]},{"osd":7,"last update":"Fri Oct 6 04:33:22 2023","interfaces":[{"interface":"back","average":{"1min":0.57099999999999995,"5min":0.56699999999999995,"15min":0.56699999999999995},"min":{"1min":0.36899999999999999,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.69399999999999995,"5min":0.75700000000000001,"15min":0.75700000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.59799999999999998,"15min":0.60399999999999998},"min":{"1min":0.41299999999999998,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.66500000000000004,"5min":0.74299999999999999,"15min":0.74299999999999999},"last":0.55100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574879,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":297820,"kb_used_data":532,"kb_used_omap":0,"kb_used_meta":297280,"kb_avail":94074020,"statfs":{"total":96636764160,"available":96331796480,"internally_reserved":0,"allocated":544768,"data_stored":235940,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":304414720},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.99099999999999999,"15min":1.0680000000000001},"min":{"1min":0.39600000000000002,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.78800000000000003,"5min":12.130000000000001,"15min":12.130000000000001},"last":0.33500000000000002},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.58499999999999996,"15min":0.58899999999999997},"min":{"1min":0.41899999999999998,"5min":0.35299999999999998,"15min":0.35299999999999998},"max":{"1min":0.77300000000000002,"5min":1.2030000000000001,"15min":1.2030000000000001},"last":0.48299999999999998}]},{"osd":1,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.52000000000000002,"5min":0.56299999999999994,"15min":0.56999999999999995},"min":{"1min":0.35499999999999998,"5min":0.35499999999999998,"15min":0.35499999999999998},"max":{"1min":0.76300000000000001,"5min":1.2250000000000001,"15min":1.2250000000000001},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.58799999999999997,"5min":0.55300000000000005,"15min":0.54700000000000004},"min":{"1min":0.378,"5min":0.34599999999999997,"15min":0.34599999999999997},"max":{"1min":0.80800000000000005,"5min":1.169,"15min":1.169},"last":0.40400000000000003}]},{"osd":2,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.55000000000000004,"15min":0.54600000000000004},"min":{"1min":0.44600000000000001,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.78700000000000003,"5min":1.236,"15min":1.236},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.52000000000000002,"5min":0.57399999999999995,"15min":0.58299999999999996},"min":{"1min":0.40100000000000002,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.71799999999999997,"5min":1.298,"15min":1.298},"last":0.56299999999999994}]},{"osd":3,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.59899999999999998,"5min":0.58699999999999997,"15min":0.58499999999999996},"min":{"1min":0.42799999999999999,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.77600000000000002,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.44600000000000001},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.63100000000000001,"15min":0.63900000000000001},"min":{"1min":0.437,"5min":0.433,"15min":0.433},"max":{"1min":0.73899999999999999,"5min":1.488,"15min":1.488},"last":0.45700000000000002}]},{"osd":4,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.53500000000000003,"5min":0.50800000000000001,"15min":0.504},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.33700000000000002},"max":{"1min":0.72599999999999998,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.57199999999999995,"5min":0.55900000000000005,"15min":0.55700000000000005},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.74399999999999999,"5min":0.77900000000000003,"15min":0.77900000000000003},"last":0.53500000000000003}]},{"osd":6,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.54900000000000004,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.39500000000000002,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.69899999999999995,"5min":0.90900000000000003,"15min":0.90900000000000003},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.58999999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"min":{"1min":0.47499999999999998,"5min":0.42899999999999999,"15min":0.42899999999999999},"max":{"1min":0.66200000000000003,"5min":0.97199999999999998,"15min":0.97199999999999998},"last":0.63900000000000001}]},{"osd":7,"last update":"Fri Oct 6 04:33:24 2023","interfaces":[{"interface":"back","average":{"1min":0.629,"5min":0.61899999999999999,"15min":0.61699999999999999},"min":{"1min":0.51400000000000001,"5min":0.45400000000000001,"15min":0.45400000000000001},"max":{"1min":0.82599999999999996,"5min":0.88800000000000001,"15min":0.88800000000000001},"last":0.57899999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.63,"15min":0.627},"min":{"1min":0.56299999999999994,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.79500000000000004,"5min":0.92600000000000005,"15min":0.92600000000000005},"last":0.54800000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-10-06T04:33:41.247 DEBUG:teuthology.orchestra.run.smithi123:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-10-06T04:33:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.364+0000 7f8a8ab30700 1 Processor -- start 2023-10-06T04:33:41.365 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.364+0000 7f8a8ab30700 1 -- start start 2023-10-06T04:33:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a8ab30700 1 --2- >> v2:172.21.15.177:3300/0 conn(0x7f8a8413f9f0 0x7f8a8406b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.369 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a8ab30700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 0x7f8a840c7660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a8ab30700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a8406b020 con 0x7f8a8413fdc0 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a8ab30700 1 -- --> v2:172.21.15.177:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a8406a450 con 0x7f8a8413f9f0 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a8ab30700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8406a6f0 con 0x7f8a8406b9f0 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a83fff700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f8a8413fdc0 0x7f8a84067660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39402/0 (socket says 172.21.15.123:39402) 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a83fff700 1 -- 172.21.15.123:0/2631822967 learned_addr learned my addr 172.21.15.123:0/2631822967 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a890cd700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 0x7f8a840c7660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a888cc700 1 --2- 172.21.15.123:0/2631822967 >> v2:172.21.15.177:3300/0 conn(0x7f8a8413f9f0 0x7f8a8406b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2730039857 0 0) 0x7f8a8406b020 con 0x7f8a8413fdc0 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 --> v1:172.21.15.123:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a74003570 con 0x7f8a8413fdc0 2023-10-06T04:33:41.370 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 <== mon.0 v1:172.21.15.123:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 156693053 0 0) 0x7f8a74003570 con 0x7f8a8413fdc0 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 msgr2=0x7f8a840c7660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 0x7f8a840c7660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 >> v2:172.21.15.177:3300/0 conn(0x7f8a8413f9f0 msgr2=0x7f8a8406b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 --2- 172.21.15.123:0/2631822967 >> v2:172.21.15.177:3300/0 conn(0x7f8a8413f9f0 0x7f8a8406b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 --> v1:172.21.15.123:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8406a9d0 con 0x7f8a8413fdc0 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 <== mon.0 v1:172.21.15.123:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 2915519208 0 0) 0x7f8a70001470 con 0x7f8a8413fdc0 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.365+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 >> v1:172.21.15.123:6789/0 conn(0x7f8a8413fdc0 legacy=0x7f8a84067660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a74003e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 0x7f8a74004810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a74004d50 0x7f8a740051c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.371 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8406a450 con 0x7f8a74003a60 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8406a6f0 con 0x7f8a74004d50 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a7400ba60 con 0x7f8a8406b9f0 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a888cc700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a74003e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a83fff700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 0x7f8a74004810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a890cd700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a74004d50 0x7f8a740051c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a888cc700 1 -- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 msgr2=0x7f8a74004810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a888cc700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a8406b9f0 0x7f8a74004810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a888cc700 1 -- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a74004d50 msgr2=0x7f8a740051c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a888cc700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a74004d50 0x7f8a740051c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a888cc700 1 -- 172.21.15.123:0/2631822967 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a74003570 con 0x7f8a74003a60 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a888cc700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a74003e70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009c00 tx=0x7f8a780091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76f3a85711f9b9 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.372 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800e6b0 con 0x7f8a74003a60 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800e810 con 0x7f8a74003a60 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.366+0000 7f8a837fe700 1 -- 172.21.15.123:0/2631822967 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a7800ebd0 con 0x7f8a74003a60 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.367+0000 7f8a8ab30700 1 -- 172.21.15.123:0/2631822967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 msgr2=0x7f8a74003e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.367+0000 7f8a8ab30700 1 --2- 172.21.15.123:0/2631822967 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a74003e70 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f8a78009c00 tx=0x7f8a780091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.367+0000 7f8a890cd700 1 -- 172.21.15.123:0/2631822967 reap_dead start 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.367+0000 7f8a8ab30700 1 -- 172.21.15.123:0/2631822967 shutdown_connections 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.367+0000 7f8a8ab30700 1 -- 172.21.15.123:0/2631822967 >> 172.21.15.123:0/2631822967 conn(0x7f8a840cd020 msgr2=0x7f8a840cd420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.367+0000 7f8a8ab30700 1 -- 172.21.15.123:0/2631822967 shutdown_connections 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.367+0000 7f8a8ab30700 1 -- 172.21.15.123:0/2631822967 wait complete. 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.368+0000 7f8a8ab30700 1 Processor -- start 2023-10-06T04:33:41.373 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.368+0000 7f8a8ab30700 1 -- start start 2023-10-06T04:33:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a8ab30700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a8415b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a8ab30700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a841593e0 0x7f8a8413aa20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a8ab30700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a84159cc0 0x7f8a8415c080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a8ab30700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8413af60 con 0x7f8a74003a60 2023-10-06T04:33:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a8ab30700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8415a610 con 0x7f8a84159cc0 2023-10-06T04:33:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a8ab30700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a8415a8f0 con 0x7f8a841593e0 2023-10-06T04:33:41.376 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a8415b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a8415b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48488/0 (socket says 172.21.15.123:48488) 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 -- 172.21.15.123:0/3545680289 learned_addr learned my addr 172.21.15.123:0/3545680289 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a890cd700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a84159cc0 0x7f8a8415c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a83fff700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a841593e0 0x7f8a8413aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 -- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a841593e0 msgr2=0x7f8a8413aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a841593e0 0x7f8a8413aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 -- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a84159cc0 msgr2=0x7f8a8415c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a84159cc0 0x7f8a8415c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 -- 172.21.15.123:0/3545680289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a8415abd0 con 0x7f8a74003a60 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a83fff700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a841593e0 0x7f8a8413aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:41.377 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a890cd700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a84159cc0 0x7f8a8415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a888cc700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a8415b6b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8a74004730 tx=0x7f8a7801a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78022070 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a7800e980 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a8415aeb0 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.369+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a78027470 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.370+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a8415a160 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.370+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a8406d240 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.371+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f8a7801c450 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.371+0000 7f8a80ff9700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8a68066f30 0x7f8a680693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.371+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f8a78085fa0 con 0x7f8a74003a60 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.371+0000 7f8a83fff700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8a68066f30 0x7f8a680693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.378 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.371+0000 7f8a83fff700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8a68066f30 0x7f8a680693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000e10 tx=0x7f8a70009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.379 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.375+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f8a780b75a0 con 0x7f8a74003a60 2023-10-06T04:33:41.560 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.559+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f8a8413af60 con 0x7f8a74003a60 2023-10-06T04:33:41.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.565+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f8a78053350 con 0x7f8a74003a60 2023-10-06T04:33:41.566 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.565+0000 7f8a80ff9700 1 -- 172.21.15.123:0/3545680289 <== mon.0 v2:172.21.15.123:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v14)=0 v14) v1 ==== 123+0+0 (secure 0 0 0) 0x7f8a78056de0 con 0x7f8a74003a60 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8a68066f30 msgr2=0x7f8a680693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8a68066f30 0x7f8a680693e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f8a70000e10 tx=0x7f8a70009040 comp rx=0 tx=0).stop 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 msgr2=0x7f8a8415b6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a8415b6b0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7f8a74004730 tx=0x7f8a7801a000 comp rx=0 tx=0).stop 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 shutdown_connections 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f8a68066f30 0x7f8a680693e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f8a74003a60 0x7f8a8415b6b0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f8a84159cc0 0x7f8a8415c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 --2- 172.21.15.123:0/3545680289 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f8a841593e0 0x7f8a8413aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.569 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 >> 172.21.15.123:0/3545680289 conn(0x7f8a840cd020 msgr2=0x7f8a840b2490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:41.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 shutdown_connections 2023-10-06T04:33:41.570 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.567+0000 7f8a8ab30700 1 -- 172.21.15.123:0/3545680289 wait complete. 2023-10-06T04:33:41.580 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-10-06T04:33:41.581 DEBUG:teuthology.orchestra.run.smithi123:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-10-06T04:33:41.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.697+0000 7f2395e3f700 1 Processor -- start 2023-10-06T04:33:41.698 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.697+0000 7f2395e3f700 1 -- start start 2023-10-06T04:33:41.700 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f2395e3f700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23901597e0 0x7f2390068670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f2395e3f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f23900c6dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f2395e3f700 1 -- --> v1:172.21.15.123:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f239006af00 con 0x7f2390159410 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f2395e3f700 1 -- --> v2:172.21.15.123:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23900c8050 con 0x7f23901597e0 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f2395e3f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f239006b460 con 0x7f2390068bb0 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238effd700 1 --2- >> v2:172.21.15.123:3300/0 conn(0x7f23901597e0 0x7f2390068670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238f7fe700 1 --1- >> v1:172.21.15.123:6789/0 conn(0x7f2390159410 0x7f23900af6a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.123:6789/0 says I am v1:172.21.15.123:39410/0 (socket says 172.21.15.123:39410) 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238f7fe700 1 -- 172.21.15.123:0/2530052867 learned_addr learned my addr 172.21.15.123:0/2530052867 (peer_addr_for_me v1:172.21.15.123:0/0) 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238ffff700 1 --2- 172.21.15.123:0/2530052867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f23900c6dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238effd700 1 -- 172.21.15.123:0/2530052867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 msgr2=0x7f23900c6dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238effd700 1 --2- 172.21.15.123:0/2530052867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f23900c6dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.701 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238effd700 1 -- 172.21.15.123:0/2530052867 >> v1:172.21.15.123:6789/0 conn(0x7f2390159410 legacy=0x7f23900af6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238effd700 1 -- 172.21.15.123:0/2530052867 --> v2:172.21.15.123:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2390068fe0 con 0x7f23901597e0 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238e7fc700 1 -- 172.21.15.123:0/2530052867 <== mon.0 v1:172.21.15.123:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3014040019 0 0) 0x7f239006af00 con 0x7f2390159410 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238effd700 1 --2- 172.21.15.123:0/2530052867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f2390068670 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f23840098e0 tx=0x7f23840091a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e33746b0f8caf1 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238e7fc700 1 -- 172.21.15.123:0/2530052867 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2384009e80 con 0x7f23901597e0 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238e7fc700 1 -- 172.21.15.123:0/2530052867 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f238400b860 con 0x7f23901597e0 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.698+0000 7f238e7fc700 1 -- 172.21.15.123:0/2530052867 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238400bc00 con 0x7f23901597e0 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 -- 172.21.15.123:0/2530052867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 msgr2=0x7f2390068670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 --2- 172.21.15.123:0/2530052867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f2390068670 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f23840098e0 tx=0x7f23840091a0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 -- 172.21.15.123:0/2530052867 shutdown_connections 2023-10-06T04:33:41.702 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 --2- 172.21.15.123:0/2530052867 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f23900c6dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 --2- 172.21.15.123:0/2530052867 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f2390068670 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 -- 172.21.15.123:0/2530052867 >> 172.21.15.123:0/2530052867 conn(0x7f23900c9c90 msgr2=0x7f239006da30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 -- 172.21.15.123:0/2530052867 shutdown_connections 2023-10-06T04:33:41.703 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.699+0000 7f2395e3f700 1 -- 172.21.15.123:0/2530052867 wait complete. 2023-10-06T04:33:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.700+0000 7f2395e3f700 1 Processor -- start 2023-10-06T04:33:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.700+0000 7f2395e3f700 1 -- start start 2023-10-06T04:33:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f2395e3f700 1 --2- >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f2390156620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f2395e3f700 1 --2- >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2390159410 0x7f23900a83e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.705 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f2395e3f700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f23900b15b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f2395e3f700 1 -- --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23900a8ce0 con 0x7f23901597e0 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f2395e3f700 1 -- --> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23900a8fc0 con 0x7f2390159410 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f2395e3f700 1 -- --> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23900a92a0 con 0x7f2390068bb0 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f23900b15b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 --2- >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f23900b15b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.123:3300/0 says I am v2:172.21.15.123:48498/0 (socket says 172.21.15.123:48498) 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 -- 172.21.15.123:0/562050104 learned_addr learned my addr 172.21.15.123:0/562050104 (peer_addr_for_me v2:172.21.15.123:0/0) 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238effd700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2390159410 0x7f23900a83e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238f7fe700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f2390156620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 -- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 msgr2=0x7f2390156620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f2390156620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.706 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 -- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2390159410 msgr2=0x7f23900a83e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2390159410 0x7f23900a83e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.701+0000 7f238ffff700 1 -- 172.21.15.123:0/562050104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384009420 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.702+0000 7f238ffff700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f23900b15b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2388007f20 tx=0x7f2388009860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.702+0000 7f23777fe700 1 -- 172.21.15.123:0/562050104 <== mon.0 v2:172.21.15.123:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f238800eb90 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.702+0000 7f23777fe700 1 -- 172.21.15.123:0/562050104 <== mon.0 v2:172.21.15.123:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f238800ed10 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.702+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23900ac590 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.702+0000 7f23777fe700 1 -- 172.21.15.123:0/562050104 <== mon.0 v2:172.21.15.123:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2388014690 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.702+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23900a8920 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.703+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23900ac3f0 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.703+0000 7f23777fe700 1 -- 172.21.15.123:0/562050104 <== mon.0 v2:172.21.15.123:3300/0 4 ==== mgrmap(e 5) v1 ==== 84740+0+0 (secure 0 0 0) 0x7f2388014a00 con 0x7f23901597e0 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.703+0000 7f23777fe700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f237c066f30 0x7f237c0693e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-10-06T04:33:41.707 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.704+0000 7f23777fe700 1 -- 172.21.15.123:0/562050104 <== mon.0 v2:172.21.15.123:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 5378+0+0 (secure 0 0 0) 0x7f2388084ab0 con 0x7f23901597e0 2023-10-06T04:33:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.704+0000 7f238f7fe700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f237c066f30 0x7f237c0693e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-10-06T04:33:41.708 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.704+0000 7f238f7fe700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f237c066f30 0x7f237c0693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2380007160 tx=0x7f2380007f50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-10-06T04:33:41.709 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.708+0000 7f23777fe700 1 -- 172.21.15.123:0/562050104 <== mon.0 v2:172.21.15.123:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+173446 (secure 0 0 0) 0x7f23900ac3f0 con 0x7f23901597e0 2023-10-06T04:33:41.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.933+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 --> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23900acab0 con 0x7f23901597e0 2023-10-06T04:33:41.935 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.934+0000 7f23777fe700 1 -- 172.21.15.123:0/562050104 <== mon.0 v2:172.21.15.123:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 30 v30) v1 ==== 94+0+3538 (secure 0 0 0) 0x7f23900acab0 con 0x7f23901597e0 2023-10-06T04:33:41.935 INFO:teuthology.orchestra.run.smithi123.stdout: 2023-10-06T04:33:41.936 INFO:teuthology.orchestra.run.smithi123.stdout:{"epoch":30,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4829,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6837/2896573989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6835","nonce":2896573989},{"type":"v1","addr":"172.21.15.123:6837","nonce":2896573989}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5068,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6833/1233807634","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6832","nonce":1233807634},{"type":"v1","addr":"172.21.15.177:6833","nonce":1233807634}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5071,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.177:6835/126227253","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.177:6834","nonce":126227253},{"type":"v1","addr":"172.21.15.177:6835","nonce":126227253}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5826,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.123:6836/2389011007","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.123:6834","nonce":2389011007},{"type":"v1","addr":"172.21.15.123:6836","nonce":2389011007}]},"join_fscid":-1,"export_targets":[],"features":4540138320759226367,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":30}],"filesystems":[]} 2023-10-06T04:33:41.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f237c066f30 msgr2=0x7f237c0693e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f237c066f30 0x7f237c0693e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f2380007160 tx=0x7f2380007f50 comp rx=0 tx=0).stop 2023-10-06T04:33:41.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 msgr2=0x7f23900b15b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-10-06T04:33:41.937 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f23900b15b0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f2388007f20 tx=0x7f2388009860 comp rx=0 tx=0).stop 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 shutdown_connections 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:6832/119418,v1:172.21.15.123:6833/119418] conn(0x7f237c066f30 0x7f237c0693e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.123:3300/0,v1:172.21.15.123:6789/0] conn(0x7f23901597e0 0x7f23900b15b0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3301/0,v1:172.21.15.177:6790/0] conn(0x7f2390068bb0 0x7f2390156620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 --2- 172.21.15.123:0/562050104 >> [v2:172.21.15.177:3300/0,v1:172.21.15.177:6789/0] conn(0x7f2390159410 0x7f23900a83e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 >> 172.21.15.123:0/562050104 conn(0x7f23900c9c90 msgr2=0x7f23900c2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 shutdown_connections 2023-10-06T04:33:41.938 INFO:teuthology.orchestra.run.smithi123.stderr:2023-10-06T04:33:41.936+0000 7f2395e3f700 1 -- 172.21.15.123:0/562050104 wait complete. 2023-10-06T04:33:41.939 INFO:teuthology.orchestra.run.smithi123.stderr:dumped fsmap epoch 30 2023-10-06T04:33:41.949 DEBUG:tasks.cephfs.filesystem:already dead... 2023-10-06T04:33:41.949 INFO:teuthology.misc:Shutting down mds daemons... 2023-10-06T04:33:41.950 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-10-06T04:33:41.950 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:41.958 INFO:tasks.ceph.mds.a:Stopped 2023-10-06T04:33:41.958 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-10-06T04:33:41.958 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.007 INFO:tasks.ceph.mds.c:Stopped 2023-10-06T04:33:42.007 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-10-06T04:33:42.007 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.018 INFO:tasks.ceph.mds.b:Stopped 2023-10-06T04:33:42.018 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-10-06T04:33:42.018 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.068 INFO:tasks.ceph.mds.d:Stopped 2023-10-06T04:33:42.069 INFO:teuthology.misc:Shutting down osd daemons... 2023-10-06T04:33:42.069 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-10-06T04:33:42.069 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.101 INFO:tasks.ceph.osd.0:Stopped 2023-10-06T04:33:42.102 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-10-06T04:33:42.102 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.166 INFO:tasks.ceph.osd.1:Stopped 2023-10-06T04:33:42.166 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-10-06T04:33:42.166 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.229 INFO:tasks.ceph.osd.2:Stopped 2023-10-06T04:33:42.229 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-10-06T04:33:42.229 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.299 INFO:tasks.ceph.osd.3:Stopped 2023-10-06T04:33:42.299 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-10-06T04:33:42.299 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.333 INFO:tasks.ceph.osd.4:Stopped 2023-10-06T04:33:42.334 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-10-06T04:33:42.334 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.407 INFO:tasks.ceph.osd.5:Stopped 2023-10-06T04:33:42.408 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-10-06T04:33:42.408 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.474 INFO:tasks.ceph.osd.6:Stopped 2023-10-06T04:33:42.474 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-10-06T04:33:42.474 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.542 INFO:tasks.ceph.osd.7:Stopped 2023-10-06T04:33:42.543 INFO:teuthology.misc:Shutting down mgr daemons... 2023-10-06T04:33:42.543 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-10-06T04:33:42.543 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.565 INFO:tasks.ceph.mgr.y:Stopped 2023-10-06T04:33:42.566 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-10-06T04:33:42.566 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.606 INFO:tasks.ceph.mgr.x:Stopped 2023-10-06T04:33:42.606 INFO:teuthology.misc:Shutting down mon daemons... 2023-10-06T04:33:42.606 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-10-06T04:33:42.607 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.617 INFO:tasks.ceph.mon.a:Stopped 2023-10-06T04:33:42.617 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-10-06T04:33:42.617 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.657 INFO:tasks.ceph.mon.b:Stopped 2023-10-06T04:33:42.657 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-10-06T04:33:42.657 INFO:teuthology.orchestra.run:waiting for 300 2023-10-06T04:33:42.707 INFO:tasks.ceph.mon.c:Stopped 2023-10-06T04:33:42.708 INFO:tasks.ceph:Checking cluster log for badness... 2023-10-06T04:33:42.708 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T04:33:42.741 INFO:teuthology.orchestra.run.smithi123.stdout:1696566683.8791435 mon.a (mon.0) 179 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T04:33:42.742 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-10-06T04:33:42.742 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T04:33:42.815 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T04:33:42.888 DEBUG:teuthology.orchestra.run.smithi123:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v 'bad backtrace' | egrep -v 'object missing on disk' | egrep -v 'error reading table object' | egrep -v 'error reading sessionmap' | egrep -v 'unmatched fragstat' | egrep -v 'unmatched rstat' | egrep -v 'was unreadable, recreating it now' | egrep -v 'Scrub error on inode' | egrep -v 'Metadata damage detected' | egrep -v MDS_FAILED | egrep -v MDS_DAMAGE | head -n 1 2023-10-06T04:33:42.961 INFO:teuthology.orchestra.run.smithi123.stdout:1696566683.8791435 mon.a (mon.0) 179 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2023-10-06T04:33:42.961 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:33:42.961 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-10-06T04:33:43.212 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:33:43.212 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-10-06T04:33:43.284 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:33:43.285 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-10-06T04:33:43.383 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:33:43.384 DEBUG:teuthology.orchestra.run.smithi123:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-10-06T04:33:43.453 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:33:43.454 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-10-06T04:33:43.688 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:33:43.688 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-10-06T04:33:43.796 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:33:43.796 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-10-06T04:33:43.869 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:33:43.869 DEBUG:teuthology.orchestra.run.smithi177:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-10-06T04:33:43.937 INFO:tasks.ceph:Archiving mon data... 2023-10-06T04:33:43.938 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/data/mon.a.tgz 2023-10-06T04:33:43.939 DEBUG:teuthology.orchestra.run.smithi123:> mktemp 2023-10-06T04:33:43.958 INFO:teuthology.orchestra.run.smithi123.stdout:/tmp/tmp.4H55sScTYe 2023-10-06T04:33:43.958 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4H55sScTYe 2023-10-06T04:33:44.092 DEBUG:teuthology.orchestra.run.smithi123:> sudo chmod 0666 /tmp/tmp.4H55sScTYe 2023-10-06T04:33:44.180 DEBUG:teuthology.orchestra.remote:smithi123:/tmp/tmp.4H55sScTYe is 306KB 2023-10-06T04:33:44.275 DEBUG:teuthology.orchestra.run.smithi123:> rm -fr /tmp/tmp.4H55sScTYe 2023-10-06T04:33:44.292 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/data/mon.b.tgz 2023-10-06T04:33:44.292 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-10-06T04:33:44.315 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.2PDrfks0r7 2023-10-06T04:33:44.315 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.2PDrfks0r7 2023-10-06T04:33:44.469 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.2PDrfks0r7 2023-10-06T04:33:44.557 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.2PDrfks0r7 is 408KB 2023-10-06T04:33:44.652 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.2PDrfks0r7 2023-10-06T04:33:44.669 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/data/mon.c.tgz 2023-10-06T04:33:44.670 DEBUG:teuthology.orchestra.run.smithi177:> mktemp 2023-10-06T04:33:44.726 INFO:teuthology.orchestra.run.smithi177.stdout:/tmp/tmp.ZD0QJUd35d 2023-10-06T04:33:44.727 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.ZD0QJUd35d 2023-10-06T04:33:44.867 DEBUG:teuthology.orchestra.run.smithi177:> sudo chmod 0666 /tmp/tmp.ZD0QJUd35d 2023-10-06T04:33:44.955 DEBUG:teuthology.orchestra.remote:smithi177:/tmp/tmp.ZD0QJUd35d is 408KB 2023-10-06T04:33:45.051 DEBUG:teuthology.orchestra.run.smithi177:> rm -fr /tmp/tmp.ZD0QJUd35d 2023-10-06T04:33:45.068 INFO:tasks.ceph:Cleaning ceph cluster... 2023-10-06T04:33:45.069 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T04:33:45.071 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-10-06T04:33:45.135 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-10-06T04:33:45.135 DEBUG:teuthology.orchestra.run.smithi123:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T04:33:45.141 DEBUG:teuthology.orchestra.run.smithi177:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-10-06T04:33:45.170 INFO:teuthology.orchestra.run.smithi123.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T04:33:45.202 INFO:teuthology.orchestra.run.smithi177.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-10-06T04:33:45.208 INFO:tasks.ceph:Archiving crash dumps... 2023-10-06T04:33:45.210 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/remote/smithi123/crash 2023-10-06T04:33:45.211 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T04:33:45.245 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/lib/ceph/crash to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/remote/smithi177/crash 2023-10-06T04:33:45.246 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-10-06T04:33:45.275 INFO:tasks.ceph:Shutting down logrotate 2023-10-06T04:33:45.276 DEBUG:teuthology.orchestra.run.smithi123:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:45.289 DEBUG:teuthology.orchestra.run.smithi177:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:45.375 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:45.404 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-10-06T04:33:45.431 INFO:tasks.ceph:Compressing logs... 2023-10-06T04:33:45.431 DEBUG:teuthology.orchestra.run.smithi123:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:33:45.446 DEBUG:teuthology.orchestra.run.smithi177:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:33:46.428 INFO:tasks.ceph:Archiving logs... 2023-10-06T04:33:46.429 DEBUG:teuthology.misc:Transferring archived files from smithi123:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/remote/smithi123/log 2023-10-06T04:33:46.430 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T04:33:46.774 DEBUG:teuthology.misc:Transferring archived files from smithi177:/var/log/ceph to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/remote/smithi177/log 2023-10-06T04:33:46.775 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /var/log/ceph -- . 2023-10-06T04:33:46.914 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-10-06T04:33:46.939 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-10-06T04:33:46.939 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T04:33:46.941 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-10-06T04:33:47.016 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-10-06T04:33:47.027 INFO:teuthology.task.clock:Checking final clock skew... 2023-10-06T04:33:47.028 DEBUG:teuthology.orchestra.run.smithi123:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:33:47.030 DEBUG:teuthology.orchestra.run.smithi177:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-10-06T04:33:47.046 INFO:teuthology.orchestra.run.smithi123.stderr:bash: ntpq: command not found 2023-10-06T04:33:47.047 INFO:teuthology.orchestra.run.smithi177.stderr:bash: ntpq: command not found 2023-10-06T04:33:47.053 INFO:teuthology.orchestra.run.smithi123.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:33:47.054 INFO:teuthology.orchestra.run.smithi123.stdout:=============================================================================== 2023-10-06T04:33:47.054 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 25 +450us[ +450us] +/- 71ms 2023-10-06T04:33:47.054 INFO:teuthology.orchestra.run.smithi123.stdout:^* hv02.front.sepia.ceph.com 3 6 377 26 -482us[ -486us] +/- 75ms 2023-10-06T04:33:47.054 INFO:teuthology.orchestra.run.smithi123.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +114us[ +110us] +/- 82ms 2023-10-06T04:33:47.054 INFO:teuthology.orchestra.run.smithi123.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:33:47.055 INFO:teuthology.orchestra.run.smithi177.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2023-10-06T04:33:47.055 INFO:teuthology.orchestra.run.smithi177.stdout:=============================================================================== 2023-10-06T04:33:47.055 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 26 +478us[ +510us] +/- 71ms 2023-10-06T04:33:47.055 INFO:teuthology.orchestra.run.smithi177.stdout:^* hv02.front.sepia.ceph.com 3 6 377 25 -498us[ -465us] +/- 75ms 2023-10-06T04:33:47.055 INFO:teuthology.orchestra.run.smithi177.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 26 +94us[ +126us] +/- 82ms 2023-10-06T04:33:47.056 INFO:teuthology.orchestra.run.smithi177.stdout:^? hv04.front.sepia.ceph.com 0 8 0 - +0ns[ +0ns] +/- 0ns 2023-10-06T04:33:47.056 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-10-06T04:33:47.065 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-10-06T04:33:47.066 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-10-06T04:33:47.110 DEBUG:teuthology.orchestra.run.smithi123:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T04:33:47.210 DEBUG:teuthology.orchestra.run.smithi177:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2023-10-06T04:33:47.303 DEBUG:teuthology.orchestra.run.smithi123:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:33:47.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:33:47.336 DEBUG:teuthology.orchestra.run.smithi177:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 2023-10-06T04:33:47.368 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:33:47.369 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-10-06T04:33:47.381 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-10-06T04:33:47.410 INFO:teuthology.task.internal:Duration was 801.143352 seconds 2023-10-06T04:33:47.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-10-06T04:33:47.461 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-10-06T04:33:47.462 DEBUG:teuthology.orchestra.run.smithi123:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T04:33:47.464 DEBUG:teuthology.orchestra.run.smithi177:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-10-06T04:33:47.509 INFO:teuthology.orchestra.run.smithi123.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:33:47.511 INFO:teuthology.orchestra.run.smithi177.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2023-10-06T04:33:47.777 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-10-06T04:33:47.777 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi123.front.sepia.ceph.com 2023-10-06T04:33:47.777 DEBUG:teuthology.orchestra.run.smithi123:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T04:33:47.800 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi177.front.sepia.ceph.com 2023-10-06T04:33:47.800 DEBUG:teuthology.orchestra.run.smithi177:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-10-06T04:33:47.824 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-10-06T04:33:47.824 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:33:47.842 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-10-06T04:33:47.904 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-10-06T04:33:47.904 DEBUG:teuthology.orchestra.run.smithi123:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T04:33:47.922 DEBUG:teuthology.orchestra.run.smithi177:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-10-06T04:33:48.013 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-10-06T04:33:48.024 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-10-06T04:33:48.024 DEBUG:teuthology.orchestra.run.smithi123:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T04:33:48.052 DEBUG:teuthology.orchestra.run.smithi177:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-10-06T04:33:48.080 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-10-06T04:33:48.097 DEBUG:teuthology.orchestra.run.smithi123:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:33:48.100 DEBUG:teuthology.orchestra.run.smithi177:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:33:48.124 INFO:teuthology.orchestra.run.smithi123.stdout:kernel.core_pattern = core 2023-10-06T04:33:48.146 INFO:teuthology.orchestra.run.smithi177.stdout:kernel.core_pattern = core 2023-10-06T04:33:48.160 DEBUG:teuthology.orchestra.run.smithi123:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:33:48.196 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:33:48.196 DEBUG:teuthology.orchestra.run.smithi177:> test -e /home/ubuntu/cephtest/archive/coredump 2023-10-06T04:33:48.217 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-10-06T04:33:48.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-10-06T04:33:48.229 INFO:teuthology.task.internal:Transferring archived files... 2023-10-06T04:33:48.230 DEBUG:teuthology.misc:Transferring archived files from smithi123:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/remote/smithi123 2023-10-06T04:33:48.230 DEBUG:teuthology.orchestra.run.smithi123:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T04:33:48.274 DEBUG:teuthology.misc:Transferring archived files from smithi177:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270/remote/smithi177 2023-10-06T04:33:48.275 DEBUG:teuthology.orchestra.run.smithi177:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-10-06T04:33:48.312 INFO:teuthology.task.internal:Removing archive directory... 2023-10-06T04:33:48.312 DEBUG:teuthology.orchestra.run.smithi123:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T04:33:48.315 DEBUG:teuthology.orchestra.run.smithi177:> rm -rf -- /home/ubuntu/cephtest/archive 2023-10-06T04:33:48.363 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-10-06T04:33:48.375 INFO:teuthology.task.internal:Not uploading archives. 2023-10-06T04:33:48.375 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-10-06T04:33:48.444 INFO:teuthology.task.internal:Tidying up after the test... 2023-10-06T04:33:48.445 DEBUG:teuthology.orchestra.run.smithi123:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T04:33:48.447 DEBUG:teuthology.orchestra.run.smithi177:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-10-06T04:33:48.462 INFO:teuthology.orchestra.run.smithi123.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 04:33 /home/ubuntu/cephtest 2023-10-06T04:33:48.464 INFO:teuthology.orchestra.run.smithi177.stdout: 525357 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Oct 6 04:33 /home/ubuntu/cephtest 2023-10-06T04:33:48.466 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-10-06T04:33:48.503 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-10-06T04:33:48.604 INFO:teuthology.nuke:Checking targets against current locks 2023-10-06T04:33:48.630 DEBUG:teuthology.nuke:shortname: smithi123 2023-10-06T04:33:48.630 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:33:48.651 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi123.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:12:16.140109', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNp26BaAn7SY4GNBkQ7fC3roi2Fy5jDHELVp1aQAnmwKkLpQ6giX1JCjpJCFE1YDQ/lLsVn0wDCnUbMkRM0TRoc='} 2023-10-06T04:33:48.657 DEBUG:teuthology.nuke:shortname: smithi177 2023-10-06T04:33:48.657 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-10-06T04:33:48.680 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi177.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/yuriw-2023-10-05_23:52:50-fs-wip-yuri-testing-2023-10-05-0730-quincy-distro-default-smithi/7412270', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '8', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-10-06 04:12:16.141973', 'locked_by': 'scheduled_yuriw@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBB2u8XPPwgdHORuEefQ8WR+QZ56B5fuFI/t8L6agrYh8Ue1xj0j0+zirQ3XAVbygcIz91+KnMCvpSIxGAfzXA54='} 2023-10-06T04:33:48.700 INFO:teuthology.orchestra.console.smithi123:Power off 2023-10-06T04:33:48.700 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T04:33:48.721 DEBUG:teuthology.orchestra.console.smithi123:power off output: Chassis Power Control: Down/Off 2023-10-06T04:33:48.721 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:33:48.736 INFO:teuthology.orchestra.console.smithi177:Power off 2023-10-06T04:33:48.737 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-10-06T04:33:48.745 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-10-06T04:33:48.758 DEBUG:teuthology.orchestra.console.smithi177:power off output: Chassis Power Control: Down/Off 2023-10-06T04:33:48.758 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:33:48.784 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-10-06T04:33:52.745 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:33:52.786 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:33:52.856 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-10-06T04:33:52.896 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-10-06T04:33:56.860 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:33:56.898 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:33:56.971 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is on 2023-10-06T04:33:57.009 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is on 2023-10-06T04:34:00.974 DEBUG:teuthology.orchestra.console.smithi123:pexpect command: ipmitool -H smithi123.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:34:01.010 DEBUG:teuthology.orchestra.console.smithi177:pexpect command: ipmitool -H smithi177.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-10-06T04:34:01.084 DEBUG:teuthology.orchestra.console.smithi123:check power output: Chassis Power is off 2023-10-06T04:34:01.120 DEBUG:teuthology.orchestra.console.smithi177:check power output: Chassis Power is off 2023-10-06T04:34:01.184 INFO:teuthology.orchestra.console.smithi123:Power off completed 2023-10-06T04:34:01.221 INFO:teuthology.orchestra.console.smithi177:Power off completed 2023-10-06T04:34:01.326 INFO:teuthology.run:Summary data: description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{centos_8} mount/fuse objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/alternate-pool} duration: 801.1433522701263 failure_reason: '"1696566683.8791435 mon.a (mon.0) 179 : cluster [WRN] Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED)" in cluster log' flavor: default owner: scheduled_yuriw@teuthology success: false 2023-10-06T04:34:01.326 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-10-06T04:34:01.407 INFO:teuthology.run:FAIL